Streaming social media event detection method based on graph clustering

Abstract

The invention relates to the technical field of natural language processing and information retrieval, in particular to a streaming social media event detection method based on graph clustering, which comprises the following steps of: firstly, segmenting a continuously input social media data stream into discrete message blocks through a predefined time window or a quantity threshold value; text information of the message blocks is processed through a pre-training language model SBERT to generate semantic vectors, and multi-modal composite node representation is formed by combining time features obtained through OLE date analysis. On the basis, a dynamic social contact message graph is constructed, an initial attribute edge is established through public attributes such as users and tags, a semantic edge is optimized through a dynamic similarity threshold value, and the stability of a graph structure is ensured. The graph representation is input in the form of an adjacent matrix and a feature matrix and is used for event detection of a graph neural network or a traditional clustering algorithm. Incremental detection is transferred to stream detection, and social event detection can be better carried out.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing and information retrieval, and in particular to a streaming social media event detection method based on graph clustering. Background Art

[0002] In today's digital age, social media platforms such as Twitter, Facebook, and Instagram have become important ways for people to communicate and share information. The massive amount of data generated by these platforms contains rich information about social events. How to effectively detect and identify social events has become a highly sought-after research topic.

[0003] In recent years, research on social event detection based on graph clustering has flourished. Currently, this approach is performed within an incremental framework: first, social media message streams are segmented into time-sequential message blocks according to preset time intervals. The messages within these blocks are then combined into a message graph, and the events contained within this message graph are independently detected. Because graph neural networks (GNNs) have powerful message aggregation capabilities and can combine the natural language representation of messages with that of their neighbors, most social event detection methods use GNN-based methods to detect events within message graphs. However, incremental event detection has significant drawbacks. In real-world scenarios, social events are continuous, and this segmentation approach significantly fragments the integrity of events. Furthermore, when constructing the graph, the top k edges with the highest similarity are selected for each node, which introduces a significant amount of noisy edges, affecting experimental results. Given the temporal nature of events, when detecting a single message block, the social message graph formed by that block will inevitably contain many isolated message nodes. These isolated message nodes are mostly from the closing messages of events in the previous block and the starting messages of new events. These nodes share no common attributes with the messages in the current block and are semantically more distant. Due to the presence of these isolated nodes, detecting events using the message graph formed by the current block may result in the misdetection of other events, thereby reducing the accuracy of event detection.

[0004] To address the aforementioned issues, the present invention proposes a new social event detection framework that is more suitable for real-world scenarios: streaming social event detection based on graph clustering. Specifically, the streaming social event detection framework receives a continuous stream of social messages, using message blocks as processing units. For each received message block, a new social message graph is constructed by combining the current message block with previously detected messages using attribute edges and semantic edges based on one-dimensional structural entropy. Over time, the number of messages to be detected increases, and a large time span can lead to semantic drift, which may prevent historical messages from being correctly assigned to the appropriate clusters. Therefore, the present invention does not perform event detection on the entire streaming social message graph. Instead, it uses a fixed-size node activity window, restricting only message nodes within the active window to be reassigned to other clusters, while message nodes outside the active window retain their original cluster structure. By precisely controlling the number of nodes in the graph, the present invention can effectively reduce computational complexity, significantly improving the scalability of the model when processing large amounts of data in real-world scenarios. This approach not only optimizes computational efficiency but also better addresses the semantic drift problem that occurs in long-term data streams, ensuring the stability and accuracy of the model over long periods of time. In addition, by introducing an active window mechanism and a dynamic graph update strategy, an efficient and scalable solution is provided for real-time event detection. This solution is particularly suitable for dynamic scenarios with a continuous influx of social media data, such as the tracking and monitoring of emergencies. Even with a dramatic increase in data volume, this method can still maintain excellent performance and fast response speed. Its "local re-clustering + global stability" approach can be transferred to other streaming analysis tasks (such as dynamic recommendation systems). The streaming social media event detection framework can make good use of the connection between historical events and messages in the current message block, ensuring the continuity of events and greatly improving the accuracy of event detection. Through these innovations, not only is incremental social media event detection extended to streaming scenarios, but the accuracy and efficiency of social event detection are also improved. It is also applicable to other tasks with streaming data characteristics, and has important practical significance and application value. Summary of the Invention

[0005] The purpose of the present invention is to provide a streaming social media event detection method based on graph clustering, which can construct a good social message graph, transfer incremental detection to streaming detection, and better perform social event detection.

[0006] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:

[0007] A method for detecting streaming social media events based on graph clustering, comprising the following steps:

[0008] S1: Segment the social media data stream into discrete message block units according to a predefined time window or quantity threshold;

[0009] S2: Use the pre-trained language model SBERT to semantically embed the text content in each message block to generate a high-dimensional semantic vector, and extract the two-dimensional time feature vector of the timestamp, and splice the two into a composite feature vector for the node;

[0010] S3: Constructing a dynamic social message graph: Establishing attribute edges based on the common attributes of messages; calculating the semantic similarity between nodes without attribute associations, discretizing the candidate threshold set, determining the optimal similarity threshold by minimizing the graph structure entropy, and constructing semantic edges;

[0011] S4: Represent the graph structure as the joint input of the adjacency matrix and the feature matrix;

[0012] S5: Use graph neural networks or traditional clustering algorithms to perform initial event detection on the graph generated by the first message block and output a set of event clusters;

[0013] S6: Incrementally update the graph structure for new message blocks, set a fixed-capacity node activity window, only perform re-clustering on the nodes within the window, and keep the cluster affiliation of nodes outside the window stable. The window slides as new messages arrive, and optimizes the incremental detection efficiency by combining historical cluster center initialization parameters.

[0014] Beneficial effects of the present invention:

[0015] The present invention effectively solves the problems of event fragmentation and high computational complexity in traditional incremental methods through a dynamic update mechanism of streaming message graphs and node activity window constraints. The present invention fuses new message blocks and historical messages into a unified graph structure through attribute edges and semantic edges based on structural entropy optimization, forming a topological connection across time blocks to ensure the integrity of the event life cycle. When a new message block arrives, the attribute edges automatically associate nodes of the same user or topic in the historical messages, while the semantic edges connect nodes with similar semantics but unrelated attributes through similarity threshold stable points, thereby constructing a dynamic graph with global semantic coherence. The active window mechanism further limits only nodes within the window to participating in re-clustering, and the cluster structure of nodes outside the window remains stable. Based on the temporal decay characteristics of event propagation: the window size covers the active period of the event, and the nodes outside the window do not need to be adjusted because they exceed the natural life cycle of the event. While reducing computational complexity, global semantic stability is maintained through local adjustments to avoid semantic drift problems caused by long-term data flow.

[0016] To address the problem that traditional similarity threshold selection is highly subjective and prone to introducing noisy edges, this invention discretizes the continuous similarity interval, calculates the one-dimensional structural entropy value of the graph structure corresponding to each candidate threshold, and selects the threshold that minimizes the entropy value as the basis for semantic edge connection. Structural entropy reflects the degree of topological disorder of the graph. Minimum entropy corresponds to the most stable connection state. Low-entropy graphs have clearer community structures, with similar nodes highly clustered and different nodes sparsely connected, thereby automatically balancing edge density and information purity. By dynamically determining the threshold by entropy minimization, we can accurately capture pairs of nodes with close semantic connections and avoid manual parameter adjustment bias.

[0017] The present invention constructs a node vector representation through a multimodal feature fusion strategy, combining text semantic embedding (SBERT model) with time feature encoding (OLE date format decomposition) to enhance the ability to characterize the dynamic evolution of events. The SBERT model generates text semantic vectors to capture deep semantic information such as message keywords and syntactic structures; the timestamp is converted into a two-dimensional vector to encode the characteristics of the event propagation stage; the node representation formed by splicing the two types of features has both semantic similarity discrimination and temporal correlation analysis capabilities; although the nodes of the same event in the fermentation period and the climax period are semantically similar, the time features are significantly different. By fusing the time dimension, the model can distinguish between messages at different stages of an event, avoiding misjudging historical event revival messages as new events. In addition, OLE time encoding retains the continuity and periodicity of time, and adapts to the periodic propagation characteristics of social media events.

[0018] The scalable hybrid clustering framework of the present invention supports the flexible integration of graph neural networks and traditional clustering algorithms to meet the trade-off requirements of efficiency and accuracy in different scenarios. GNN aggregates neighbor node information through multi-layer graph convolution to generate globally semantically consistent node embeddings; while lightweight algorithms such as k-means can quickly divide the local cluster structure within the active window. The combination of the two forms a streaming detection paradigm of "global semantic guidance + local efficient adjustment". GNN is suitable for complex events that are sensitive to global topology, capturing long-range semantic dependencies through multi-layer aggregation; while k-means uses historical cluster center initialization to accelerate convergence when processing incremental messages within the window. In addition, the active window mechanism limits the data size of each re-clustering, so that the system can dynamically switch to a lightweight algorithm when resources are limited to ensure real-time response. In the breaking news monitoring scenario, the combination of GNN + windowed k-means achieves a processing throughput of hundreds of messages per second with a delay of less than 2 seconds, meeting the real-time analysis needs of high-concurrency streaming data.

[0019] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A graph clustering-based streaming social event detection framework diagram provided by an embodiment of the present invention;

[0022] The highlighted portion indicates that the complete event in the incremental framework has been dissected, resulting in isolated nodes that may be mistakenly detected as other events, reducing the accuracy of social event detection. The streaming framework can effectively integrate historical events to manage complete social events, making up for the shortcomings of the incremental framework and enabling more efficient and accurate social event detection.

[0023] Figure 2 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0025] Example 1

[0026] The method for detecting streaming social media events based on graph clustering described in this embodiment includes the following steps:

[0027] S1. For incoming social media message streams, use message blocks as a processing method. Whenever a social stream reaches a preset message block size or contains messages within a certain period of time, it is processed into a message block for social event detection.

[0028] S2. Use the pre-trained language model SBERT (Sentence Bidirectional Encoder Representations from Transformers) to perform sentence embedding on the messages in each message block to obtain a message similarity vector representation. Convert the timestamp of each message to OLE date format to generate a two-dimensional time feature vector containing integer and decimal parts. Concatenate the message embedding vector and the time feature vector to represent the message node in the social message graph.

[0029] S3. The first block of messages is used to form a social message graph, where a social message represents a node in the social message graph. Messages typically have common attributes such as users (including the user itself and users mentioned in the message), hashtags, and named entities. Message nodes with the same attribute values are connected using attribute edges to increase the relevance between messages. Message nodes that describe the same event but do not have the same attribute values and have high similarity are considered to have semantic edges. The similarity range is divided into n discrete similarity values. For each similarity value, the entropy of the social message graph is calculated. The similarity threshold with the minimum entropy is found as the similarity threshold stabilization point of the social message graph. The semantic edges in the social message graph are connected based on this stabilization point.

[0030] S4. Based on the vector representation of the message nodes obtained in step S2 and the edges obtained in step S3, the edge relationships between messages in the social message graph can be represented by an adjacency matrix, and the vector representation of each message node can be represented by a two-dimensional array.

[0031] S5. Use a social event detection method (such as a GNN-based method or a traditional clustering algorithm such as k-means) to perform event detection on the social message graph obtained from the first block of messages;

[0032] S6. For continuously arriving message blocks, the messages in the current message block and historical messages are reconstructed into a message graph according to the S3 method. A node activity window is set. As new message blocks arrive, the activity window moves forward. Events detected before the node activity window are considered relatively stable and do not need to be detected again. Only the message nodes within the activity window are detected. This process continues until all message blocks in the social stream are detected. For each stage, a clustering result with multiple clusters is obtained using the social event detection algorithm. The clustering effect is evaluated using indicators such as NMI (Normalized Mutual Information), AMI (Adjusted Mutual Information), and ARI (Adjusted Rand Index).

[0033] In this embodiment, the S1 specifically includes:

[0034] Define social media message stream: S = M0, M1, ..., M i-1 ,M i ,… is a set of social media message blocks in a continuous time series, M i In the time block Time block = [t i , t i+1 ) constitutes a social media message block consisting of all messages in M i Expressed as: M i ={m j |1≤i≤|Mi |}, where |M i |For M i The total number of messages in m j For specific messages; m j Expressed as: m j ={doc j ,u j ,nm j ,h j ,t j}, where doc j ,u j ,nm j ,h j ,t j Represent the relevant text documents, users (senders and mentioned users), named entities, hashtags and timestamps respectively; the same type of social media messages (assuming that each social media message belongs to only one category) is represented as: e = {m i |1≤i≤|e|}, where |e| represents the total number of messages in e;

[0035] In this embodiment, S2 specifically includes:

[0036] Input the text of each message into the pre-trained language model Sentence-BERT to obtain the text embedding representation d j _feature=SBERT_embed(doc j ), doc j ∈m j ; Convert the timestamp of each message into a vector t j _feature=time_embed(t j ), t j ∈m j ; The text embedding representation of the message is concatenated with the temporal feature representation of the message to obtain the vector representation m of the message j _feature=np.concatenet(d j _feature,t j _feature);

[0037] In this embodiment, S3 specifically includes:

[0038] For message nodes with the same attribute value, edges are used to connect them to increase the relevance between messages. The resulting edges are called attribute edges E. a For message nodes that describe the same event but do not have the same attribute values and have high similarity, they are considered to have a semantic edge E s ; Use cosine similarity s i,j =cos(mi ,m j ) measures the similarity between messages, if s i,j >0.6, it is considered that there is a strong correlation between the messages, and semantic edges are added between the messages; if s i,j <0.3, it is considered that the messages have a very weak correlation and no semantic edge is added; for s i,j ∈(0.3,0.6), and consider their correlation strength to be fuzzy; divide the continuous values between (0.3, 0.6) into 30 discrete values thr={x|x=0.3+0.01n,n∈Z,0≤n≤30} at intervals of 0.01; respectively, according to the similarity value thr i Add semantic edges and calculate the similarity value thr i The entropy value of the corresponding social message graph:

[0039]

[0040] Where V represents all message nodes, |V| represents the number of message nodes, d j represents the degree (weight) of node j, and vol(λ) represents the total degree of the social graph; calculate the average of these entropy values:

[0041]

[0042] Taking τ as the final similarity threshold stable point, for the similarity s i,j >τ messages, add semantic edges;

[0043] In this embodiment, the S4 specifically includes:

[0044] The attribute edge E obtained above a and semantic edge E s As the edge of the entire social graph G, we get the social graph G = (V, E), E = E a ∪E s The attribute edge E obtained above a and semantic edge E s As the edge of the entire social message graph G, we get the social message graph G = (V, E), E = E a ∪E s , the edge relationship between graph message nodes is represented by the adjacency matrix:

[0045]

[0046] Where A represents the adjacency matrix, N represents the number of message nodes, a i,j =1 indicates that there is a connection between message node i and message node j, otherwise a i,j =0; the vector representation matrix of the message node is:

[0047]

[0048] in is the vector representation of the i-th message node, and d represents the length of the vector obtained after each message is embedded in the pre-trained language model;

[0049] In this embodiment, S5 specifically includes:

[0050] Use mainstream methods to detect events in social message graphs within a streaming framework. For example, methods based on graph neural networks (GNNs) use multi-layer graph convolutions to gradually integrate node embeddings with information from neighboring nodes, forming a more globally semantically accurate node representation. The message node embeddings obtained through GNNs are then clustered using k-means or DBSCAN, ultimately yielding a clustering result consisting of many clusters.

[0051] In this embodiment, S6 specifically includes:

[0052] For the i-th message block M in the social stream S i , M i The messages in the network and the historical messages form a new social message graph G'=(V',E'), where V'=V∪V i , V i Indicates M i The message node in V represents the historical message node, and E' represents M i Attribute edge E' between the message and the historical message a and semantic edge E' s , E'={(m k ,m j )|m k ,m j ∈V'}; Set a node activity window w_size and use the message node in the detection activity window The clustering result C' is obtained. The event cluster C detected before the active window remains unchanged. The clustering result C'∪C obtained for all message nodes is evaluated using indicators such as NMI (Normalized Mutual Information), AMI (Adjusted Mutual Information) and ARI (Adjusted Rand Index).

[0053] Example 2

[0054] Using the Events2012 and Events2018 datasets, after filtering out duplicate and unavailable tweets, Events2012 contains 68,841 English tweets and 503 social events. Events2018 contains 64,516 French tweets and 257 social events. The two-dimensional structural entropy minimization algorithm (EUSEvent) was used as a graph clustering algorithm in the streaming social event detection framework. The average metrics of all message blocks were calculated, and the results shown in Table 1 demonstrate the feasibility of the proposed method.

[0055] Table 1 Average clustering index of Events2012 and Events2018

[0056]

[0057] In summary, the present invention proposes a method for streaming social media event detection based on graph clustering. First, the continuously input social media data stream is divided into discrete message blocks through a predefined time window or quantity threshold. The text information of these message blocks is processed by the pre-trained language model SBERT to generate semantic vectors, and combined with the time features obtained by OLE date parsing to form a multimodal composite node representation. On this basis, a dynamic social message graph is constructed, and initial attribute edges are established through common attributes such as users and tags. The semantic edges are optimized through dynamic similarity thresholds to ensure the stability of the graph structure. The graph representation is input in the form of an adjacency matrix and a feature matrix for event detection using graph neural networks or traditional clustering algorithms. In order to maintain the detection effect of incremental data, an active window constraint is introduced, and only the nodes within the window are re-clustered in real time, combining historical data to optimize efficiency. This method significantly improves the accuracy and real-time performance of event detection through technologies such as semantic edge optimization driven by structural entropy.

[0058] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for detecting streaming social media events based on graph clustering, characterized in that: The following steps are involved: S1: Segment the social media data stream into discrete message block units according to a predefined time window or quantity threshold; S2: Use the pre-trained language model SBERT to semantically embed the text content in each message block to generate a high-dimensional semantic vector, and extract the two-dimensional time feature vector of the timestamp, and splice the two into a composite feature vector for the node; S3: Constructing a dynamic social message graph: Establishing attribute edges based on the common attributes of messages; calculating the semantic similarity between nodes without attribute associations, discretizing the candidate threshold set, determining the optimal similarity threshold by minimizing the graph structure entropy, and constructing semantic edges; S4: Represent the graph structure as the joint input of the adjacency matrix and the feature matrix; S5: Use graph neural networks or traditional clustering algorithms to perform initial event detection on the graph generated by the first message block and output a set of event clusters; S6: Incrementally update the graph structure for new message blocks, set a fixed-capacity node activity window, only perform re-clustering on the nodes within the window, and keep the cluster affiliation of nodes outside the window stable. The window slides as new messages arrive, and optimizes the incremental detection efficiency by combining historical cluster center initialization parameters.

2. The method for detecting streaming social media events based on graph clustering according to claim 1, wherein: Said S1 specifically includes: Define social media message stream: S = M0, M1, ..., M i-1 ,M i ,… is a set of social media message blocks in a continuous time series, M i In the time block Timeblock=[t i , t i+1 ) constitutes a social media message block consisting of all messages in M i Expressed as: M i ={m j |1≤i≤|M i |}, where |M i |For M i The total number of messages in m j For specific messages; m j Expressed as: m j ={doc j ,u j ,nm j ,h j ,t j }, where doc j ,u j ,nm j ,h j ,t j Represent related text documents, users, named entities, hashtags and timestamps respectively; the same type of social media messages are represented as: e = {m i |1≤i≤|e|}, where |e| represents the total number of messages in e.

3. The method for detecting streaming social media events based on graph clustering according to claim 1, wherein: The S2 specifically includes: Input the text of each message into the pre-trained language model Sentence-BERT to obtain the text embedding representation d j _feature=SBERT_embed(doc j ), doc j ∈m j ; Convert the timestamp of each message into a vector t j _feature=time_embed(t j ), t j ∈m j ; The text embedding representation of the message is concatenated with the temporal feature representation of the message to obtain the vector representation m of the message j _feature=np.concatenet(d j _feature,t j _feature).

4. The method for detecting streaming social media events based on graph clustering according to claim 1, wherein: The S3 specifically includes: For message nodes with the same attribute value, edges are used to connect them to increase the relevance between messages. The resulting edges are called attribute edges E. a For message nodes that describe the same event but do not have the same attribute values and have high similarity, they are considered to have a semantic edge E s ; Use cosine similarity s i,j =cos(m i ,m j ) measures the similarity between messages, if s i,j >0.6, it is considered that there is a strong correlation between the messages, and semantic edges are added between the messages; if s i,j <0.3, it is considered that the messages have a very weak correlation and no semantic edge is added; for s i,j ∈(0.3,0.6), and consider their correlation strength to be fuzzy; divide the continuous values between (0.3, 0.6) into 30 discrete values thr={x|x=0.3+0.01n,n∈Z,0≤n≤30} at intervals of 0.01; respectively, according to the similarity value thr i Add semantic edges and calculate the similarity value thr i The entropy value of the corresponding social message graph: Where V represents all message nodes, |V| represents the number of message nodes, d j represents the degree (weight) of node j, and vol(λ) represents the total degree of the social graph; calculate the average of these entropy values: Taking τ as the final similarity threshold stable point, for the similarity s i,j >τ messages, add semantic edges.

5. The method for detecting streaming social media events based on graph clustering according to claim 1, wherein: The S4 specifically includes: The attribute edge E obtained above a and semantic edge E s As the edge of the entire social graph G, we get the social graph G = (V, E), E = E a ∪E s The attribute edge E obtained above a and semantic edge E s As the edge of the entire social message graph G, we get the social message graph G = (V, E), E = E a ∪E s , the edge relationship between graph message nodes is represented by the adjacency matrix: Where A represents the adjacency matrix, N represents the number of message nodes, and a i,j =1 indicates that there is a connection between message node i and message node j, otherwise a i,j =0; the vector representation matrix of the message node is: in is the vector representation of the i-th message node, and d represents the length of the vector obtained after each message is embedded in the pre-trained language model.

6. The method for detecting streaming social media events based on graph clustering according to claim 1, wherein: The S5 specifically includes: Mainstream methods are used to detect events in social message graphs within a streaming framework. For example, methods based on graph neural networks (GNNs) use multi-layer graph convolutions to gradually integrate node embeddings with information from neighboring nodes, thereby forming a node representation with more global semantics. The message node embeddings obtained through GNNs are then clustered using k-means or DBSCAN, ultimately yielding a clustering result consisting of many clusters.

7. The method for detecting streaming social media events based on graph clustering according to claim 1, wherein: The S6 specifically includes: For the i-th message block M in the social stream S i , M i The messages in the network and the historical messages form a new social message graph G'=(V',E'), where V'=V∪V i , V i Indicates M i The message node in V represents the historical message node, and E' represents M i Attribute edge E' between the message and the historical message a and semantic edge E' s , E'={(m k ,m j )|m k ,m j ∈V'}; Set a node activity window w_size and use the message node in the detection activity window The clustering result C' is obtained, the event cluster C detected before the active window remains unchanged, and the clustering result C'∪C obtained for all message nodes is used to evaluate the clustering effect using NMI, AMI and ARI indicators.

Images