Hot event dynamic monitoring method and device based on public information

Abstract

The invention relates to the technical field of hotspot monitoring, in particular to a hotspot event dynamic monitoring method and device based on public information.The method comprises the steps that a public information text set is obtained, word segmentation operation is conducted on each public information text in the public information text set, and a word segmentation sequence of the public information texts is obtained, performing hot word extraction based on the word segmentation sequence to obtain hot words of the word segmentation sequence, namely a hot word set of the public information text set; and performing hot word graph modeling on the hot word set based on a frequent item set to obtain a hot word graph structure of the hot word set. According to the method, community detection is carried out on the hot word graph, so that the relationship and aggregation trend between the hot words can be deeply analyzed, and the association degree between different hot words and the community to which the hot words belong can be identified, and therefore, the evolution process and the internal structure of the event can be understood from multiple dimensions, and complex social events can be better understood.

Description

Technical Field

[0001] This invention relates to the field of hotspot monitoring technology, specifically to a method and apparatus for dynamic monitoring of hotspot events based on publicly available information. Background Technology

[0002] Public information text sets refer to all publicly accessible text data collections, which can come from various public information sources such as news, social media, and forums.

[0003] Traditional methods often rely on manual screening and analysis, resulting in slow discovery and response times for trending events, especially in rapidly changing environments such as social media and news websites. Traditional methods may fail to capture the latest trending developments in real time. Furthermore, they typically rely on simple keyword matching and manual analysis, which is often susceptible to noise and interference, making it difficult to accurately identify truly important events and topics. Traditional methods also tend to analyze trending events from a single angle or level, focusing only on superficial hot words while ignoring deeper underlying events or the evolution of events from different perspectives. Moreover, traditional methods often emphasize single-dimensional monitoring, such as simple keyword frequency statistics, lacking analysis of text similarity and deep relationships. This may lead to missing different content or related events connected to the trending event. Finally, traditional methods struggle to deeply understand the evolutionary process and internal structure of events, often remaining only at the surface level of trending topics. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method and device for dynamic monitoring of hot events based on publicly available information.

[0005] The technical solution adopted to solve the above-mentioned technical problems is: a method for dynamic monitoring of hot events based on publicly available information, including:

[0006] Obtain a set of publicly available information texts, perform word segmentation on each publicly available information text in the set to obtain a word segmentation sequence of the publicly available information texts, and extract hot words based on the word segmentation sequence to obtain the hot words of the word segmentation sequence, that is, obtain the hot word set of the publicly available information text set;

[0007] The hot word set is modeled using frequent itemsets to obtain the hot word graph structure. Hot word communities are detected using the hot word graph structure based on modularity optimization to obtain the hot word community set of the hot word graph structure.

[0008] Cluster the hot words in each hot word community in the hot word community set to obtain a sub-hot word community set, and obtain the public information text subset corresponding to the sub-hot word community set;

[0009] Calculate the similarity between any two public information texts in the subset of public information texts, and add the two public information texts with similarity higher than a preset similarity threshold to the hot event monitoring set, thereby obtaining the hot event monitoring set corresponding to the subset of public information texts;

[0010] The number of publicly available information texts in the hot event monitoring set is obtained. When the number exceeds a preset threshold, a hot event monitoring alarm is generated.

[0011] Prioritize hot word extraction based on the segmented word sequence to obtain the hot words of the segmented word sequence, including:

[0012] The contextual semantic features and word embedding vectors of the segmented sequence are obtained based on the trained BERT model;

[0013] The similarity between each word in the segmented sequence is calculated based on the word embedding vector of the segmented sequence. If the similarity is greater than a preset similarity threshold, it indicates that there is a similar relationship between the two words.

[0014] A directed graph is constructed based on the similarity relationship between the word segmentation sequence and the words. The directed graph is aggregated based on graph convolution operation to obtain the text hidden features of the word segmentation sequence. The directed graph includes a set of nodes and a set of directed edges. The nodes correspond to the words in the word segmentation sequence, and the directed edges correspond to the similarity relationship between the two words.

[0015] The contextual semantic features of the word segmentation sequence and the textual hidden features are fused to obtain fused features. The fused features are then classified to obtain the hot words of the word segmentation sequence.

[0016] Preferredly, the directed graph is aggregated based on graph convolution operations to obtain the text-hidden features of the word segmentation sequence, including:

[0017] The directed graph is embedded based on graph convolution operations to obtain node representations, wherein the formula for the node representation is as follows:

[0018]

[0019] Among them, H l+1 Let σ represent the hidden representation of the (l+1)th layer node in a directed graph, and let σ() represent the activation function. The column represents the adjacency matrix of a directed graph of type τ. The column represents the hidden representation of the l-th level nodes in a directed graph of type τ. The column represents the weight of the l-th layer node in a directed graph of type τ, where T represents the different types of nodes;

[0020] Based on the node representation of each node in the directed graph, the type-level attention score of adjacent nodes is calculated, wherein the formula for calculating the type-level attention score is as follows:

[0021]

[0022] Among them, a τ μ represents the type-level attention score of adjacent nodes. τ The column represents an attention vector of type τ, || represents the concatenation operation, and h v h represents the hidden representation of the current node. τ The column represents the embedding representation of neighboring nodes of type τ, and Indicates adjacent nodes. This represents the embedding of the directed edges between the current node and its neighboring nodes. This represents the hidden representation of adjacent nodes.

[0023] Prioritizing the aggregation of the directed graph based on graph convolution operations to obtain the text-hidden features of the word segmentation sequence, the method further includes:

[0024] The type-level attention weights of neighboring nodes are obtained based on their type-level attention scores. The formula for calculating the type-level attention weights is as follows:

[0025]

[0026] Where, α τ This represents the type-level attention weights;

[0027] The node-level attention score of the adjacent nodes is calculated based on the type-level attention weights of the adjacent nodes, wherein the calculation formula for the node-level attention score is as follows:

[0028]

[0029] in, The attention weights, v, represent the type-level attention weights of neighboring nodes. T Represents the attention vector;

[0030] The textual hidden features of the segmented sequence are obtained by fusing a two-level attention mechanism that combines type-level and node-level attention.

[0031] Prioritizes performing hot word graph modeling on the hot word set based on frequent itemsets to obtain the hot word graph structure of the hot word set, including:

[0032] Frequently occurring keywords are extracted from the publicly available information text set to obtain the keyword set of the publicly available information text set. The judgment of frequent occurrence is specifically that the number of times a word appears is higher than a preset occurrence threshold.

[0033] The frequent itemset extraction is performed on the keyword set based on the frequent itemset algorithm to obtain the frequent itemsets of the public information text set;

[0034] A hot word graph structure is constructed based on the hot word set and the frequent itemset. The hot word graph structure includes a set of hot word nodes and a set of undirected edges. The hot word nodes correspond to the hot words, and the undirected edges are used to connect two hot words. When two hot words appear in the frequent itemset at the same time, the two hot words are connected by the undirected edges.

[0035] Prioritize performing hot word community detection on the hot word graph structure based on modularity optimization to obtain the hot word communities of the hot word graph structure, including:

[0036] Initialize each hot word node in the hot word graph structure as a separate community. For each node, calculate the change in modularity after moving the hot word node to the community to which the neighboring hot word nodes belong.

[0037] If the modularity increases after moving, then move the hot word node; otherwise, maintain the original community division and repeat the above operation until no hot word node can increase the modularity by moving.

[0038] Each community is treated as a new super node. The edge weights between the level nodes are calculated, and the above operation is repeated until the modularity no longer increases, so as to obtain the hot word community set of the hot word graph structure.

[0039] Preferred, the formula for calculating the post-modularity of the community is as follows:

[0040]

[0041] Where Q represents the post-modularity of the community, A ij The elements in the adjacency matrix of the hot word graph structure represent whether there is an edge between hot word node i and hot word node j. i Let m represent the degree of hot word node i, and m represent the total number of edges in the hot word graph structure. i ,c j ) represents an indicator function; if hot word node i and hot word node j belong to the same community, then (c i ,c j ) = 1, otherwise (c i ,c j ) = 0.

[0042] Prioritize calculating the similarity between any two publicly available information texts in the subset of publicly available information texts, including:

[0043] The two publicly available information texts are segmented to obtain a first word sequence and a second word sequence;

[0044] The first word sequence and the second word sequence are encoded based on the Word2vec model to obtain the first word vector matrix of the first word sequence and the second word vector matrix of the second word sequence;

[0045] The first word vector matrix and the second word vector matrix are interacted to obtain the first interactive attention matrix corresponding to the first word vector matrix and the second interactive attention matrix corresponding to the second word vector matrix;

[0046] The first word vector matrix and the first interaction attention matrix are concatenated to obtain a first concatenated matrix, and the second word vector matrix and the second interaction attention matrix are concatenated to obtain a second concatenated matrix.

[0047] Preferredly, calculating the similarity between any two publicly available information texts in the subset of publicly available information texts further includes:

[0048] The first concatenation matrix and the second concatenation matrix are respectively input into the Transformer model, and the first text feature of the first concatenation matrix and the second text feature of the second concatenation matrix are output based on the Transformer model;

[0049] The first text features and the second text features are one-dimensionalized based on the fully connected layer to obtain the first semantic features and the second semantic features.

[0050] Calculate the difference and product of the first semantic feature and the second semantic feature, and concatenate the difference and product to obtain the fused feature;

[0051] The fused features are processed using a two-layer fully connected network to obtain the similarity between two publicly available information texts. The first layer of the fully connected network uses the ReLU activation function, and the second layer of the connected network uses the Softmax normalization function.

[0052] The technical solution adopted to solve the above-mentioned technical problems is: a dynamic monitoring device for hot events based on publicly available information, which is applicable to the aforementioned dynamic monitoring method for hot events based on publicly available information, comprising:

[0053] A hot word extraction unit is used to acquire a set of publicly available information texts, perform word segmentation on each publicly available information text in the set of publicly available information texts to obtain a word segmentation sequence of the publicly available information texts, and extract hot words based on the word segmentation sequence to obtain the hot words of the word segmentation sequence, that is, to obtain the hot word set of the publicly available information text set.

[0054] The community detection unit is used to perform hot word graph modeling on the hot word set based on frequent itemsets to obtain the hot word graph structure of the hot word set, and to perform hot word community detection on the hot word graph structure based on modularity optimization to obtain the hot word community set of the hot word graph structure.

[0055] A hot word clustering unit is used to cluster the hot words in each hot word community in the hot word community set to obtain a sub-hot word community set of the hot word community and obtain the public information text subset corresponding to the sub-hot word community set;

[0056] A hotspot monitoring unit is used to calculate the similarity between any two public information texts in the subset of public information texts, and add the two public information texts with a similarity higher than a preset similarity threshold to the hotspot event monitoring set, thereby obtaining the hotspot event monitoring set corresponding to the subset of public information texts.

[0057] An event alarm unit is used to obtain the number of publicly available information texts in the hot event monitoring set, and generate a hot event monitoring alarm when the number exceeds a preset threshold.

[0058] The beneficial effects of the present invention are as follows: (1) By acquiring publicly available information text sets, the present invention can capture the latest information from public platforms such as social media and news websites in real time, ensuring the timeliness of hot topic monitoring. Through word segmentation and hot word extraction, the present invention can quickly identify current hot topics, significantly improving the real-time performance of monitoring. Furthermore, through hot word graph modeling based on frequent itemsets and hot word community detection with modularity optimization, the present invention can accurately capture the correlation between hot words and identify potential hot topics. This accurate modeling capability can effectively reduce the interference of noise data and improve the accuracy of monitoring. (2) Through hot word community detection and sub-hot word community clustering, the present invention can detect hot words from different levels and angles. Identifying hot events, this multi-level detection mechanism can capture diverse hot events, avoid the limitations of a single perspective, ensure the comprehensiveness of monitoring, and by calculating the similarity between publicly available information texts, it can identify multiple texts related to hot events, further enriching the content of hot events. This multi-dimensional similarity calculation mechanism can ensure the diversity and comprehensiveness of hot event identification; (3) By performing community detection on hot word maps, this invention can deeply analyze the relationship and aggregation trend between hot words, identify the degree of association between different hot words and the communities to which they belong, so as to understand the evolution process and internal structure of events from multiple dimensions, and better understand complex social events. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the overall method steps in one embodiment of the present invention;

[0060] Figure 2 This is a schematic diagram of the overall device architecture in one embodiment of the present invention.

[0061] Attached labels: 1. Hot word extraction unit; 2. Community detection unit; 3. Hot word clustering unit; 4. Hotspot monitoring unit; 5. Event alarm unit. Detailed Implementation

[0062] Example 1, as Figure 1 As shown, the present invention proposes a method for dynamic monitoring of hot topics based on publicly available information, comprising:

[0063] S1. Obtain the set of publicly available information texts, perform word segmentation on each publicly available information text in the set to obtain the word segmentation sequence of the publicly available information texts, and extract hot words based on the word segmentation sequence to obtain the hot words of the word segmentation sequence, that is, obtain the hot word set of the publicly available information text set.

[0064] S2. Based on frequent itemsets, perform hot word graph modeling on the hot word set to obtain the hot word graph structure of the hot word set. Based on modularity optimization, perform hot word community detection on the hot word graph structure to obtain the hot word community set of the hot word graph structure.

[0065] S3. Cluster the hot words in each hot word community in the hot word community set to obtain the sub-hot word community set, and obtain the public information text subset corresponding to the sub-hot word community set;

[0066] S4. Calculate the similarity between any two public information texts in the public information text subset, and add the two public information texts with similarity higher than the preset similarity threshold to the hot event monitoring set, thereby obtaining the hot event monitoring set corresponding to the public information text subset;

[0067] S5. Obtain the number of publicly available information texts in the hot event monitoring center. When the number exceeds the preset threshold, generate a hot event monitoring alarm.

[0068] In this invention, the publicly available information text set refers to all publicly accessible text data sets, which can come from various public information sources such as news, social media, and forums; word segmentation is the process of breaking down a piece of text into smaller units (usually words or phrases). For example, word segmentation of Chinese text divides a sentence into different words. Word segmentation is an important step in text preprocessing, helping computers understand the specific meaning of the text; hot words refer to words that frequently appear and have a certain importance within a certain time period or a specific text set; in data mining, frequent itemsets refer to items (or words) that frequently appear together in a given dataset. The analysis of frequent itemsets can help identify words or phrases that frequently appear together in text, thereby understanding hot topics or events; modularity is an indicator for measuring the effectiveness of network partitioning, it reflects... The optimization of modularity reflects the rationality of the graph division. This is achieved by adjusting the graph division to ensure strong connections between nodes within the same module and weaker connections between different modules. This operation is used to discover different communities in the hot word graph. Within each hot word community, there may be smaller, more granular sub-communities. Through clustering methods, hot words in the hot word community can be further subdivided into multiple sub-communities, reflecting more detailed and specific hot event content. The hot event monitoring set contains a collection of publicly available information texts related to a certain hot event. The texts in this set indicate that a certain event is receiving widespread attention or discussion. When the number of times a hot event is detected in publicly available information texts exceeds a preset threshold, the system will trigger an alarm to remind relevant personnel that the attention given to a certain hot event is rising sharply.

[0069] Example 2: The present invention proposes a method for dynamic monitoring of hot events based on publicly available information. Compared with Example 1, this example further includes: extracting hot words based on word segmentation sequences to obtain hot words from the word segmentation sequences, including:

[0070] A1. Obtain the contextual semantic features and word embedding vectors of the segmented sequence based on the pre-trained BERT model;

[0071] A2. The word embedding vector based on the word segmentation sequence is used to calculate the similarity between each word in the word segmentation sequence. If the similarity is greater than the preset similarity threshold, it indicates that there is a similar relationship between the two words.

[0072] A3. Construct a directed graph based on the similarity relationship between word segments and words, and pass the aggregated directed graph based on graph convolution operation to obtain the text hidden features of the word segments. The directed graph includes a set of nodes and a set of directed edges. The nodes correspond to the words in the word segments and the directed edges correspond to the similarity relationship between two words.

[0073] A4. The contextual semantic features and textual hidden features of the word segmentation sequence are fused to obtain fused features. The fused features are then classified to obtain the hot words of the word segmentation sequence.

[0074] In an optional embodiment, a directed graph is passed based on graph convolution operations to obtain text-hidden features of the word segmentation sequence, including:

[0075] B1. An embedding representation of a directed graph is performed based on graph convolution operations to obtain a node representation, where the formula for the node representation is as follows:

[0076]

[0077] Among them, H l+1 Let σ represent the hidden representation of the (l+1)th layer node in a directed graph, and let σ() represent the activation function. The column represents the adjacency matrix of a directed graph of type τ. The column represents the hidden representation of the l-th level nodes in a directed graph of type τ. The column represents the weight of the l-th layer node in a directed graph of type τ, where T represents the different types of nodes;

[0078] B2. Calculate the type-level attention score of neighboring nodes based on the node representation of each node in the directed graph. The formula for calculating the type-level attention score is as follows:

[0079]

[0080] Among them, a τ μ represents the type-level attention score of adjacent nodes. τ The column represents an attention vector of type τ, || represents the concatenation operation, and h v h represents the hidden representation of the current node. τ The column represents the embedding representation of neighboring nodes of type τ, and Indicates adjacent nodes. This represents the embedding of the directed edges between the current node and its neighboring nodes. This represents the hidden representation of adjacent nodes.

[0081] It's important to note that BERT is a pre-trained language model based on Transformers. It captures contextual information through a bidirectional encoder, rather than simply reading text sequentially from left to right or right to left. The BERT model learns the contextual semantic features of words and generates word embedding vectors representing the meaning of words in a specific context. Contextual semantic features refer to the semantic expression of each word in a particular context. These features depend not only on the word's intrinsic meaning but are also influenced by surrounding words. For example, the meaning of "bank" differs in "bank by the river" and "financial bank," and BERT can capture this difference. Feature fusion refers to combining multiple feature information into a unified representation. In this example, it involves fusing the contextual semantic features of the word segmentation sequence with the textual hidden features to obtain a more comprehensive feature representation. This enhances the model's ability to capture the meaning of text.

[0082] In an optional embodiment, the method of passing an aggregated directed graph based on graph convolution operations to obtain text-hidden features of the word segmentation sequence further includes:

[0083] B3. Obtain the type-level attention weights of neighboring nodes based on their type-level attention scores. The formula for calculating the type-level attention weights is as follows:

[0084]

[0085] Where, α τ This represents the type-level attention weights;

[0086] B4. Calculate the node-level attention score of neighboring nodes based on the type-level attention weights of neighboring nodes. The formula for calculating the node-level attention score is as follows:

[0087]

[0088] in, The attention weights, v, represent the type-level attention weights of neighboring nodes. T Represents the attention vector;

[0089] B5. The dual attention mechanism of type-level and node-level attention is fused to obtain the hidden text features of the word segmentation sequence.

[0090] It should be noted that graph convolution is one of the core operations in graph neural networks (GNNs). It learns node representations by combining the feature information of a node with the feature information of its neighboring nodes. The adjacency matrix is ​​a matrix used to represent the relationships between nodes in a graph. In a directed graph, the adjacency matrix is ​​a square matrix, and the elements of the matrix indicate whether there is an edge between the nodes and its direction. The attention mechanism is a technique that enables neural networks to dynamically adjust their attention based on the importance of the input data. The attention mechanism allows the model to selectively focus on the information of certain nodes when aggregating the information of neighboring nodes.

[0091] In an optional embodiment, hot word graph modeling is performed on the hot word set based on frequent itemsets to obtain the hot word graph structure of the hot word set, including:

[0092] C1. Extract frequently occurring keywords from the publicly available information text set to obtain the keyword set of the publicly available information text set. The judgment of frequent occurrence is specifically that the number of times a word appears is higher than a preset occurrence threshold.

[0093] C2. Extract frequent itemsets from the keyword set based on the frequent itemset algorithm to obtain the frequent itemsets of the public information text set;

[0094] C3. Construct a hot word graph structure based on the hot word set and frequent itemset. The hot word graph structure includes a set of hot word nodes and a set of undirected edges. Hot word nodes correspond to hot words, and undirected edges are used to connect two hot words. When two hot words appear in the frequent itemset at the same time, they are connected by undirected edges.

[0095] It should be noted that frequently occurring keywords refer to words that appear frequently in a set of text data. These words may be keywords related to the text topic or words that express specific information. By statistically analyzing the frequency of words in a text set, these frequently occurring keywords can be found. The frequent itemset algorithm is a data mining algorithm used to discover items (or features) that frequently appear together from a large amount of data. Items are usually keywords, phrases, or words, while frequent itemsets refer to combinations of words that frequently appear together in multiple texts. Common frequent itemset algorithms include the Apriori algorithm and the FP-Growth algorithm.

[0096] In an optional embodiment, hot word community detection is performed on the hot word graph structure based on modularity optimization to obtain the hot word communities of the hot word graph structure, including:

[0097] D1. Initialize each hot word node in the hot word graph structure as a separate community. For each node, calculate the change in modularity after moving the hot word node to the community to which the neighboring hot word nodes belong.

[0098] D2. If the modularity increases after moving, then move the hot word node; otherwise, maintain the original community division and repeat the above operation until no hot word node can increase the modularity by moving.

[0099] D3. Treat each community as a new super node, calculate the edge weights between super nodes, and repeat the above operation until the modularity no longer increases, so as to obtain the hot word community set of the hot word graph structure.

[0100] It's important to note that in graph theory, a community refers to a subgraph where the density of connections between a group of nodes is high, while the density of connections with other nodes in the graph is low. In other words, a community is a collection of nodes with strong relationships between them and weak relationships with other nodes in the graph. Modularity is an indicator of the quality of community partitioning in a graph. The higher the modularity, the more reasonable the community partitioning of the graph, meaning that the nodes in the graph are more concentrated in their respective communities, and there are fewer connections between communities. The modularity value is usually between -1 and 1, with a larger value indicating a better community partitioning. Supernodes refer to treating what was originally a community as a separate node. Each supernode represents an original community, and the edge weights between them reflect the connection relationships between the original communities. Through this process, the scale of the graph is reduced, and further community partitioning calculations are performed. Edge weight refers to the weight attached to each edge in the graph, usually representing the strength or importance of the connection between nodes.

[0101] In an optional embodiment, the formula for calculating the post-modularity of the community is as follows:

[0102]

[0103] Where Q represents the post-modularity of the community, A ij The elements in the adjacency matrix of the hot word graph structure represent whether there is an edge between hot word node i and hot word node j. i Let m represent the degree of hot word node i, and m represent the total number of edges in the hot word graph structure. i ,c j ) represents an indicator function; if hot word node i and hot word node j belong to the same community, then (c i ,c j ) = 1, otherwise (c i ,c j ) = 0.

[0104] In an optional embodiment, calculating the similarity between any two public information texts in a subset of public information texts includes:

[0105] E1. Perform word segmentation on the two publicly available information texts to obtain the first word sequence and the second word sequence;

[0106] E2. Encode the first word sequence and the second word sequence based on the Word2vec model to obtain the first word vector matrix of the first word sequence and the second word vector matrix of the second word sequence;

[0107] E3. Interact with the first word vector matrix and the second word vector matrix to obtain the first interactive attention matrix corresponding to the first word vector matrix and the second interactive attention matrix corresponding to the second word vector matrix;

[0108] E4. Concatenate the first word vector matrix and the first interaction attention matrix to obtain the first concatenated matrix. Concatenate the second word vector matrix and the second interaction attention matrix to obtain the second concatenated matrix.

[0109] In an optional embodiment, calculating the similarity between any two publicly available information texts in a subset of publicly available information texts further includes:

[0110] E5. Input the first concatenation matrix and the second concatenation matrix into the Transformer model respectively, and output the first text feature of the first concatenation matrix and the second text feature of the second concatenation matrix based on the Transformer model;

[0111] E6. The first text features and the second text features are one-dimensionalized based on the fully connected layer to obtain the first semantic features and the second semantic features.

[0112] E7. Calculate the difference and product of the first semantic feature and the second semantic feature, and concatenate the difference and product to obtain the fused feature;

[0113] E8. The fused features are processed based on a two-layer fully connected network to obtain the similarity between two publicly available information texts. The first layer of the fully connected network uses the ReLU activation function, and the second layer of the connected network uses the Softmax normalization function.

[0114] It should be noted that the Softmax function is a function that transforms input values ​​into a probability distribution; a fully connected network is a structure composed of two layers of neural networks, designed to further process the fused features to obtain the final output.

[0115] Example 3, as Figure 2 As shown, the present invention proposes a dynamic monitoring device for hot events based on publicly available information, which is applicable to the aforementioned dynamic monitoring method for hot events based on publicly available information, comprising:

[0116] Hot word extraction unit 1 is used to obtain a set of public information texts, perform word segmentation on each public information text in the set of public information texts to obtain a word segmentation sequence of public information texts, and extract hot words based on the word segmentation sequence to obtain the hot words of the word segmentation sequence, that is, to obtain the hot word set of the public information text set.

[0117] Community detection unit 2 is used to perform hot word graph modeling on the hot word set based on frequent itemsets to obtain the hot word graph structure of the hot word set, and to perform hot word community detection on the hot word graph structure based on modularity optimization to obtain the hot word community set of the hot word graph structure;

[0118] Hot word clustering unit 3 is used to cluster the hot words in each hot word community in the hot word community set to obtain the sub-hot word community set of the hot word community and obtain the public information text subset corresponding to the sub-hot word community set;

[0119] Hotspot monitoring unit 4 is used to calculate the similarity between any two public information texts in the public information text subset, and add two public information texts with a similarity higher than a preset similarity threshold to the hotspot event monitoring set, thereby obtaining the hotspot event monitoring set corresponding to the public information text subset;

[0120] Event Alarm Unit 5 is used to obtain the number of publicly available information texts in the hot event monitoring center. When the number exceeds a preset threshold, a hot event monitoring alarm is generated.

[0121] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A method for dynamic monitoring of hotspot events based on public information, characterized in that, The method comprises the following steps: obtaining a public information text set, performing word segmentation on each public information text in the public information text set to obtain a word segmentation sequence of the public information text, performing hot word extraction based on the word segmentation sequence to obtain a hot word of the word segmentation sequence, i.e. obtaining a hot word set of the public information text set; modeling the hot word set based on a frequent item set to obtain a hot word graph structure of the hot word set, and performing hot word community detection on the hot word graph structure based on modularity optimization to obtain a hot word community set of the hot word graph structure; clustering the hot words in each hot word community in the hot word community set to obtain a sub-hot word community set of the hot word community, and obtaining a public information text subset corresponding to the sub-hot word community set; calculating the similarity between any two public information texts in the public information text subset, adding two public information texts with a similarity higher than a preset similarity threshold to a hot event monitoring set, thereby obtaining a hot event monitoring set corresponding to the public information text subset; when the number of public information texts in the hot event monitoring set is greater than a preset number threshold, generating a hot event monitoring alarm.

2. The method of claim 1, wherein the method further comprises: The hot word extraction based on the word segmentation sequence to obtain a hot word of the word segmentation sequence comprises: obtaining the context semantic features and the word embedding vectors of the word segmentation sequence based on a trained BERT model; calculating the similarity between each word in the word segmentation sequence based on the word embedding vectors of the word segmentation sequence to obtain the similarity between each word in the word segmentation sequence, and if the similarity is greater than a preset similarity threshold, it indicates that there is a similar relationship between the two words; constructing a directed graph based on the word segmentation sequence and the similar relationship between the words, and performing graph convolution operation to aggregate the directed graph to obtain the text hidden features of the word segmentation sequence, wherein the directed graph comprises a set of nodes and a set of directed edges, wherein the nodes correspond to the words in the word segmentation sequence, and the directed edges correspond to the similar relationship between the two words; fusing the context semantic features and the text hidden features of the word segmentation sequence to obtain fusion features, and classifying the fusion features to obtain the hot word of the word segmentation sequence.

3. The hotspot event dynamic monitoring method based on public information according to claim 2, characterized in that, The graph convolution operation is used to aggregate the directed graph to obtain the text hidden features of the word segmentation sequence, which comprises: embedding representation of the directed graph based on the graph convolution operation to obtain node representation, wherein the formula of the node representation is as follows: where H l+1 denotes the hidden representation of the (l+1)th layer node of the directed graph, σ() denotes an activation function, denotes the adjacency matrix of the directed graph of type τ, denotes the hidden representation of the lth layer node of the directed graph of type τ, denotes the weight of the lth layer node of the directed graph of type τ, T denotes different types of nodes; calculating the type-level attention score of adjacent nodes based on the node representation of each node in the directed graph, wherein the calculation formula of the type-level attention score is as follows: where α τ denotes the type-level attention score of a neighboring node, μ τ denotes the attention vector of type τ, || denotes the concatenation operation, h v denotes the hidden representation of the current node, h τ denotes the embedding representation of a neighboring node of type τ, and denotes a neighboring node, denotes the embedding representation of the directed edge between the current node and the neighboring node, denotes the hidden representation of the neighboring node.

4. The hotspot event dynamic monitoring method based on public information according to claim 3, characterized in that, The graph convolution operation is used to aggregate the directed graph to obtain the text hidden features of the word segmentation sequence, which further comprises: obtaining the attention weight of the type level of adjacent nodes based on the type-level attention score of adjacent nodes, wherein the calculation formula of the attention weight of the type level is as follows: wherein, a τ denotes the attention weight of the type level; calculating a node-level attention score of the adjacent node based on an attention weight of a type level of the adjacent node, where a calculation formula of the node-level attention score is as follows: wherein, denotes an attention weight for the type level of the neighboring node, v T denotes an attention vector; fusing the type-level attention and the node-level attention to obtain a text hidden feature of the word segmentation sequence.

5. The method of claim 1, wherein the method further comprises: modeling a hot word graph based on the frequent item set to obtain a hot word graph structure of the hot word set, including: extracting frequently occurring keywords from the set of public information texts to obtain a keyword set of the set of public information texts, where the frequent occurrence is determined according to whether a number of occurrences of a word is higher than a preset occurrence threshold; extracting a frequent item set from the keyword set based on a frequent item set algorithm to obtain a frequent item set of the set of public information texts; constructing a hot word graph structure based on the hot word set and the frequent item set, where the hot word graph structure includes a set of hot word nodes and a set of undirected edges, the hot word nodes correspond to the hot words, and the undirected edges are used to connect two hot words, and when the two hot words appear in the frequent item set at the same time, the two hot words are connected by an undirected edge.

6. The hotspot event dynamic monitoring method based on public information according to claim 5, characterized in that, detecting hot word communities in the hot word graph structure based on modularity optimization to obtain hot word communities of the hot word graph structure, including: initializing each hot word node in the hot word graph structure as a separate community, and for each node, calculating a change in post-modularity of moving the hot word node to a community to which a neighbor hot word node belongs; if the modularity increases after the movement, the hot word node is moved, otherwise, the original community division is maintained, and the above operation is repeated until no movement of the hot word node can increase the modularity; each community is regarded as a new super node, the edge weight between the super nodes is calculated, and the above operation is repeated until the modularity no longer increases, to obtain a set of hot word communities of the hot word graph structure.

7. The hotspot event dynamic monitoring method based on public information according to claim 6, characterized in that, a calculation formula of the post-modularity of the community is as follows: where Q represents the post-modularity of the community, A ij represents an element in the adjacency matrix of the hotword graph structure, used to represent whether there is an edge between hotword node i and hotword node j, d i represents the degree of hotword node i, m represents the total number of edges in the hotword graph structure, (c i ,c j ) represents an indicator function, if hotword node i and hotword node j belong to the same community, (c i ,c j ) = 1, otherwise (c i ,c j ) = 0.

8. The method of claim 1, wherein the method further comprises: calculating a similarity between any two public information texts in the set of public information texts, including: performing a word segmentation operation on the two public information texts to obtain a first word sequence and a second word sequence; encoding the first word sequence and the second word sequence based on a Word2vec model to obtain a first word vector matrix of the first word sequence and a second word vector matrix of the second word sequence; interacting the first word vector matrix and the second word vector matrix to obtain a first interaction attention matrix corresponding to the first word vector matrix and a second interaction attention matrix corresponding to the second word vector matrix; performing matrix splicing on the first word vector matrix and the first interaction attention matrix to obtain a first splicing matrix, and performing matrix splicing on the second word vector matrix and the second interaction attention matrix to obtain a second splicing matrix.

9. The hotspot event dynamic monitoring method based on public information according to claim 8, characterized in that, calculating the similarity between any two public information texts in the set of public information texts, further including: The first splicing matrix and the second splicing matrix are respectively input into a Transformer model, and first text features of the first splicing matrix and second text features of the second splicing matrix are output based on the Transformer model; The first text features and the second text features are one-dimensionalized based on a full connection layer to obtain first semantic features and second semantic features; The difference and the product of the first semantic features and the second semantic features are calculated, and the difference and the product are spliced to obtain fusion features; The fusion features are processed based on a two-layer full connection network to obtain the similarity between two public information texts, wherein the first layer full connection network adopts a ReLU activation function, and the second layer connection network adopts a Softmax normalization function.

10. A hotspot event dynamic monitoring device based on public information, which is suitable for a hotspot event dynamic monitoring method based on public information according to any one of claims 1-9, characterized in that, Comprise: A hot word extraction unit (1) is used to obtain a public information text set, perform a word segmentation operation on each public information text in the public information text set to obtain a word segmentation sequence of the public information text, and perform hot word extraction based on the word segmentation sequence to obtain hot words of the word segmentation sequence, i.e., a hot word set of the public information text set; A community detection unit (2) is used to model a hot word graph based on the hot word set to obtain a hot word graph structure of the hot word set, and perform hot word community detection on the hot word graph structure based on modularity optimization to obtain a hot word community set of the hot word graph structure; A hot word clustering unit (3) is used to cluster hot words in each hot word community in the hot word community set to obtain a sub-hot word community set of the hot word community, and obtain a public information text subset corresponding to the sub-hot word community set; A hot spot monitoring unit (4) is used to calculate the similarity between any two public information texts in the public information text subset, add two public information texts with a similarity higher than a preset similarity threshold to a hot spot event monitoring set, and thereby obtain a hot spot event monitoring set corresponding to the public information text subset; An event alarm unit (5) is used to obtain the number of public information texts in the hot spot event monitoring set, and generate a hot spot event monitoring alarm when the number is greater than a preset number threshold.

Images