Dynamic situation propagation prediction system for urban social governance

Abstract

The invention discloses a dynamic event propagation prediction system for urban social governance, which is used for accurately classifying and analyzing dynamic event propagation of urban social governance, identifying early propagation signs of events in time and predicting the propagation range, influence and possible evolution paths of the events. The system comprises a public opinion information acquisition subsystem, a public opinion data center, a public opinion data preprocessing subsystem and a public opinion analysis and prediction subsystem, the public opinion information acquisition subsystem is used for performing directional information acquisition on websites in a specified area and a specified field, and the public opinion data preprocessing subsystem is used for performing code recognition, webpage noise removal, text content extraction, Chinese word segmentation and automatic abstract generation on the acquired information and then providing the information to the public opinion analysis and prediction subsystem; and the public opinion analysis and prediction subsystem is used for carrying out keyword frequency analysis, topic detection and tracking and sensitive topic identification on the preprocessed information.

Description

Technical Field

[0001] The present invention relates to the technical field of public opinion analysis applications, and in particular to a system for predicting the spread of dynamic events in urban social governance. Background Art

[0002] Non-traditional security threats such as terrorism, cyber security, major infectious diseases, and climate change are becoming increasingly prominent. Exploring the aggregation, coupling, and interactive expressions of online groups in the virtual and real society and the law of the spread of online public opinion on non-traditional security incidents has far-reaching significance for promoting the modernization of the national governance system and governance capabilities. At present, the traditional intelligent analysis methods of public opinion analysis are insufficient, and there are also problems such as interference from noise, false information, and subjective tendencies. The shortcomings such as insufficient comprehensive consideration of factors cannot meet the needs of timely and accurate grasp of the dynamics of the spread of events. It can be seen that how to solve the problem of insufficient existing intelligent analysis methods of public opinion and interference from noise, false information, and subjective tendencies has become one of the problems that need to be solved urgently. Summary of the invention

[0003] The present invention provides a system for predicting the spread of dynamic events in urban social governance, which is used to accurately classify and analyze the spread of dynamic events in urban social governance, timely identify early signs of the spread of events, predict the spread scope, influence and possible evolution path of events, help relevant departments and decision makers quickly understand the development trend of events, formulate scientific and effective response strategies and measures, and further take measures to avoid the expansion and escalation of events.

[0004] First, a system for predicting the spread of dynamic events in urban social governance is provided, including: a public opinion information collection subsystem, a public opinion data center, a public opinion data preprocessing subsystem, and a public opinion analysis and prediction subsystem;

[0005] The public opinion information collection subsystem is used to carry out targeted collection of website information in designated areas and designated fields of concern to users according to the user's configuration definition; the collected public opinion information is stored in the public opinion data center in the form of web page text to form a network public opinion information database;

[0006] The public opinion data preprocessing subsystem is used to perform coding recognition and text extraction processing on the public opinion information in the network public opinion information database, obtain the preprocessed public opinion information and transmit it to the public opinion analysis and prediction subsystem;

[0007] The public opinion analysis and prediction subsystem is used to convert the text information in the preprocessed public opinion information into a text feature vector, perform keyword frequency analysis, and determine the keywords in the text information; based on the keywords in the text information, perform topic detection and tracking on the text feature vector, and through text clustering and classification, mine and determine topic clusters, track and report on them, and record the entire process of topic germination, development, and extinction; based on the keywords in the text information, match and filter each topic cluster with sensitive keywords, identify and determine sensitive topics, and issue warnings.

[0008] In one embodiment, the public opinion information collection subsystem is specifically used to carry out targeted collection of public opinion information data of various public opinion carriers according to the user's configuration definition, automatically collect Internet web page information in various encoding formats, automatically collect the information of each collection website by encoding and identifying it based on the collection rules configured by each collection website, and perform deduplication and filtering according to the site uniform resource locator URL; the collected public opinion information is stored in the public opinion data center in the form of web page text to form a network public opinion information database.

[0009] In one embodiment, the public opinion data preprocessing subsystem is specifically used to encode and identify public opinion information in the online public opinion information database, remove web page noise, extract text content and perform Chinese word segmentation, generate an automatic summary, and obtain preprocessed public opinion information.

[0010] In one embodiment, the public opinion analysis and prediction subsystem is specifically used to convert the text information in the preprocessed public opinion information into a text feature vector; calculate the TF-IDF weight of the keyword frequency analysis of the text information; and determine the keywords in the text information based on the TF-IDF weight of the text information.

[0011] In one implementation, the public opinion analysis and prediction subsystem is specifically used to calculate the TF-IDF weight of text information using the following formula:

[0012]

[0013] TF-IDF(w)=TF(d,w)*IDF(w)

[0014] Where TF(d,w) is the frequency of word w in the text information appearing in document d, count(d,w) is the number of times word w in the text information appears in document d, count(d,*) is the total number of words in document d, IDF(w) is the inverse document frequency of word w in the text information, N is the total number of documents in the corpus, N(w) is the number of documents in which word w appears in the document, and TF-IDF(w) is the TF-IDF weight of word w in the text information.

[0015] In one embodiment, the public opinion analysis and prediction subsystem is specifically used to process the text in the text feature vector in the order of acquisition time based on the keywords in the text information, use the first text as a seed to create a first topic cluster, and sequentially determine whether the maximum similarity between the remaining texts and the existing topic clusters is greater than a preset threshold; if greater, the text is classified into the topic cluster corresponding to the maximum similarity; if not, the text is used as a seed to create a new topic cluster; follow-up reports are conducted on each topic cluster to record the entire process of topic germination, development, and extinction.

[0016] In one embodiment, the public opinion analysis and prediction subsystem is specifically used to process the text in the text feature vector in the order of acquisition time based on the keywords in the text information, use the first text as a seed to create a first topic cluster, and determine the feature center of the topic cluster, and sequentially determine whether the maximum similarity between the remaining texts and the feature center of the existing topic cluster is greater than a preset threshold; if greater, classify the text into the topic cluster corresponding to the maximum similarity, and update the feature center of the topic cluster; if not greater, use the text as a seed to create a new topic cluster, and determine the feature center of the new topic cluster; conduct follow-up reports on each topic cluster to record the entire process of topic germination, development, and extinction.

[0017] In one embodiment, the public opinion analysis and prediction subsystem is specifically used to match and filter each topic cluster with sensitive keywords based on the keywords in the text information, determine the topic sensitivity according to the weights of each part of the text information; judge whether the topic sensitivity is greater than a preset sensitivity threshold; if greater, determine it as a sensitive topic and issue a warning; if less, determine it as a general topic.

[0018] In one embodiment, the public opinion analysis and prediction subsystem is specifically used to match and filter each topic cluster with sensitive keywords based on the keywords in the text information, and determine the topic sensitivity according to the weight of each part of the text information through the following formula:

[0019]

[0020] Among them, S is the topic sensitivity, nt is the number of sensitive words contained in the text information title, Nt is the total number of words in the text information title, ns is the number of sensitive words contained in the text information summary, Ns is the total number of words in the text information summary, a is the text information title weight, and b is the text information summary weight.

[0021] In one embodiment, the system further includes a public opinion application management subsystem;

[0022] The public opinion application management subsystem is used to provide an interactive interface between the system and the user, display the prediction results of the public opinion analysis and prediction subsystem, and display the entire process of the germination, development, and extinction of various topics as well as the basic information of various topics;

[0023] The public opinion data center includes an online public opinion information database, an online public opinion analysis database, a system management database, and a sensitive word database; among them, the online public opinion information database is used to store and manage the collected data of the public opinion information collection subsystem using a distributed file system database; the online public opinion analysis database is used to store and manage the prediction result data of the public opinion analysis and prediction subsystem using a relational database, and provide it to the public opinion application management subsystem for display on the interactive interface; the system management database is used to store various configuration information of the system; the sensitive word database is used to store various predefined sensitive keywords, and the sensitive keywords are graded according to scale and nature.

[0024] An embodiment of the present invention provides a system for predicting the dissemination of dynamic events in urban social governance. The public opinion information collection subsystem is used to carry out targeted information collection on websites in designated areas and designated fields. The public opinion data preprocessing subsystem is used to encode and identify the collected information, remove web page noise, extract the main text content, and provide it to the public opinion analysis and prediction subsystem after Chinese word segmentation and automatic summary generation. The public opinion analysis and prediction subsystem is used to perform keyword frequency analysis, topic detection and tracking, and sensitive topic identification on the preprocessed information.

[0025] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0027] Figure 1 A schematic diagram of the system architecture of a city-wide social governance dynamic event propagation prediction system according to an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of the processing flow of a system for predicting the propagation of dynamic events in urban social governance according to an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of a process for collecting online public opinion information according to an embodiment of the present invention;

[0030] Figure 4 A flowchart of a keyword frequency analysis algorithm according to an embodiment of the present invention;

[0031] Figure 5 The figure is a flowchart of a sensitive topic identification algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to achieve accurate classification and analysis of the spread of dynamic events in urban social governance, timely identify early signs of the spread of events, and predict the spread, influence and possible evolution path of events, an embodiment of the present invention provides a system for predicting the spread of dynamic events in urban social governance.

[0033] It should be noted that the terms "first", "second", etc. in the description and claims of the embodiments of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0034] The preferred embodiments of the present invention are described below in conjunction with the drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other if there is no conflict.

[0035] Embodiment 1

[0036] The embodiment of the present invention provides a system for predicting the spread of dynamic events in urban social governance, including: a public opinion information collection subsystem, a public opinion data center, a public opinion data preprocessing subsystem, and a public opinion analysis and prediction subsystem;

[0037] The public opinion information collection subsystem is used to carry out targeted collection of website information in designated areas and designated fields of concern to users according to the user's configuration definition; the collected public opinion information is stored in the public opinion data center in the form of web page text to form a network public opinion information database;

[0038] The public opinion data preprocessing subsystem is used to perform coding recognition and text extraction processing on the public opinion information in the network public opinion information database, obtain the preprocessed public opinion information and transmit it to the public opinion analysis and prediction subsystem;

[0039] The public opinion analysis and prediction subsystem is used to convert the text information in the preprocessed public opinion information into a text feature vector, perform keyword frequency analysis, and determine the keywords in the text information; based on the keywords in the text information, perform topic detection and tracking on the text feature vector, and through text clustering and classification, mine and determine topic clusters, track and report on them, and record the entire process of topic germination, development, and extinction; based on the keywords in the text information, match and filter each topic cluster with sensitive keywords, identify and determine sensitive topics, and issue warnings.

[0040] In one embodiment, the public opinion information collection subsystem is specifically used to carry out targeted collection of public opinion information data of various public opinion carriers according to the user's configuration definition, automatically collect Internet web page information in various encoding formats, automatically collect the information of each collection website by encoding and identifying it based on the collection rules configured by each collection website, and perform deduplication and filtering according to the site uniform resource locator URL; the collected public opinion information is stored in the public opinion data center in the form of web page text to form a network public opinion information database.

[0041] In one embodiment, the public opinion data preprocessing subsystem is specifically used to encode and identify public opinion information in the online public opinion information database, remove web page noise, extract text content and perform Chinese word segmentation, generate an automatic summary, and obtain preprocessed public opinion information.

[0042] In one embodiment, the public opinion analysis and prediction subsystem is specifically used to convert the text information in the preprocessed public opinion information into a text feature vector; calculate the TF-IDF weight of the keyword frequency analysis of the text information; and determine the keywords in the text information based on the TF-IDF weight of the text information.

[0043] In one implementation, the public opinion analysis and prediction subsystem is specifically used to calculate the TF-IDF weight of text information using the following formula:

[0044]

[0045] TF-IDF(w)=TF(d,w)*IDF(w)

[0046] Where TF(d,w) is the frequency of word w in the text information appearing in document d, count(d,w) is the number of times word w in the text information appears in document d, count(d,*) is the total number of words in document d, IDF(w) is the inverse document frequency of word w in the text information, N is the total number of documents in the corpus, N(w) is the number of documents in which word w appears in the document, and TF-IDF(w) is the TF-IDF weight of word w in the text information.

[0047] In one embodiment, the public opinion analysis and prediction subsystem is specifically used to process the text in the text feature vector in the order of acquisition time based on the keywords in the text information, use the first text as a seed to create a first topic cluster, and sequentially determine whether the maximum similarity between the remaining texts and the existing topic clusters is greater than a preset threshold; if greater, the text is classified into the topic cluster corresponding to the maximum similarity; if not, the text is used as a seed to create a new topic cluster; follow-up reports are conducted on each topic cluster to record the entire process of topic germination, development, and extinction.

[0048] In one embodiment, the public opinion analysis and prediction subsystem is specifically used to process the text in the text feature vector in the order of acquisition time based on the keywords in the text information, use the first text as a seed to create a first topic cluster, and determine the feature center of the topic cluster, and sequentially determine whether the maximum similarity between the remaining texts and the feature center of the existing topic cluster is greater than a preset threshold; if greater, classify the text into the topic cluster corresponding to the maximum similarity, and update the feature center of the topic cluster; if not greater, use the text as a seed to create a new topic cluster, and determine the feature center of the new topic cluster; conduct follow-up reports on each topic cluster to record the entire process of topic germination, development, and extinction.

[0049] In one embodiment, the public opinion analysis and prediction subsystem is specifically used to match and filter each topic cluster with sensitive keywords based on the keywords in the text information, determine the topic sensitivity according to the weights of each part of the text information; judge whether the topic sensitivity is greater than a preset sensitivity threshold; if greater, determine it as a sensitive topic and issue a warning; if less, determine it as a general topic.

[0050] In one embodiment, the public opinion analysis and prediction subsystem is specifically used to match and filter each topic cluster with sensitive keywords based on the keywords in the text information, and determine the topic sensitivity according to the weight of each part of the text information through the following formula:

[0051]

[0052] Among them, S is the topic sensitivity, nt is the number of sensitive words contained in the text information title, Nt is the total number of words in the text information title, ns is the number of sensitive words contained in the text information summary, Ns is the total number of words in the text information summary, a is the text information title weight, and b is the text information summary weight.

[0053] In one embodiment, the system further includes a public opinion application management subsystem;

[0054] The public opinion application management subsystem is used to provide an interactive interface between the system and the user, display the prediction results of the public opinion analysis and prediction subsystem, and display the entire process of the germination, development, and extinction of various topics as well as the basic information of various topics;

[0055] The public opinion data center includes an online public opinion information database, an online public opinion analysis database, a system management database, and a sensitive word database; among them, the online public opinion information database is used to store and manage the collected data of the public opinion information collection subsystem using a distributed file system database; the online public opinion analysis database is used to store and manage the prediction result data of the public opinion analysis and prediction subsystem using a relational database, and provide it to the public opinion application management subsystem for display on the interactive interface; the system management database is used to store various configuration information of the system; the sensitive word database is used to store various predefined sensitive keywords, and the sensitive keywords are graded according to scale and nature.

[0056] For specific implementation, see Figure 1 The system includes a public opinion information collection subsystem, a public opinion data center, a public opinion data preprocessing subsystem, a public opinion analysis and prediction subsystem, and also includes a basic platform, an application support platform, and a public opinion application management subsystem. Figure 2 .

[0057] The public opinion information collection subsystem is used to carry out targeted information collection for websites in designated areas and fields. Through the link relationship between web pages, the Internet is regarded as a directed graph, starting from a series of seed sites and continuously expanding to the entire network along the link direction, and automatically obtaining the page information of each site crawled through. According to the specific needs of users, the public opinion information collection subsystem combines manual intervention with automatic collection to focus on websites in designated areas and fields that users are concerned about, and conducts precise targeted information collection. The collected information is stored in the public opinion data center in the form of web page text for subsequent statistics, analysis, and prediction.

[0058] The public opinion data preprocessing subsystem is used to preprocess the collected information data, including code recognition and text extraction. The collected web page text is coded and recognized, web page noise is removed, the text content is extracted, and after Chinese word segmentation and automatic summary generation, it is provided to the public opinion analysis and prediction subsystem for further processing.

[0059] Furthermore, the public opinion information collection subsystem is used to collect various data sources related to the city's social governance events, such as social media data, news reports, public opinion, etc. Data collection can be carried out through application programming API interfaces, web crawlers, etc. The public opinion data preprocessing subsystem is used to preprocess the collected data, including text cleaning, denoising, standardization, etc., to ensure the quality and consistency of the data.

[0060] For details, please refer to the online public opinion information collection process. Figure 3 The public opinion information collection subsystem collects public opinion information data from various public opinion carriers of domestic mainstream media and local media, such as news websites, forums, blogs, comments and microblogs, based on the user's configuration definition, and realizes the automatic collection of Internet web page information in various encoding formats to form a network public opinion information database. The network public opinion information collection subsystem supports the collection of information from various types of web pages, including static web pages, AJAX dynamic web pages (Asynchronous JavaScript and XML dynamic web pages), and other types of web pages (simple information aggregation RSS, extensible markup language XML). At the same time, it can realize the collection of information from sites that need to be logged in through site configuration, and realize real-time and comprehensive data collection of target monitoring sites.

[0061] Furthermore, the public opinion information collection subsystem configures corresponding collection rules for each collection site (collection rules include site uniform resource locator URL, filtering rules, collection frequency, etc.) to ensure the accuracy of collected web page information; the public opinion information collection subsystem encodes and identifies the website information according to the collection rule library, and performs duplicate removal and filtering according to the URL; the web page data collected by the public opinion information collection subsystem is saved in the network public opinion information database, and is called by the public opinion data preprocessing subsystem for the next step of data processing.

[0062] The public opinion analysis and prediction subsystem is used to perform topic detection and tracking, hot topic identification, sensitive topic identification and other analysis and processing on the pre-processed information. Through data mining and automatic detection, it can discover sensitive topics contained in various types of online public opinion information such as news and blogs on the Internet, and identify subsequent reports on various sensitive topics, so as to achieve full tracking of the birth, development and demise of topics. For various topics that have been discovered, based on the authority of the news source, the number of reports, the number of comments, the growth rate of the number, the time density of comments and other factors, the hot Internet public opinion topics within the user-specified time period are identified. For various topics that have been discovered, based on the premise of semantic analysis, keyword control is used to discover sensitive topics and issue corresponding alarms.

[0063] Furthermore, the public opinion analysis and prediction subsystem performs keyword frequency analysis on the preprocessed information to evaluate the importance of words to a document set or a document in the corpus. The importance of the text information is calculated for subsequent data mining processes.

[0064] Specifically, the keyword frequency analysis algorithm process can be found in Figure 4 The public opinion analysis and prediction subsystem converts the text information preprocessed by the public opinion data preprocessing subsystem into a text feature vector, and represents the text information as a vector that can express the text semantics, which is to use a numerical vector to represent the text semantics. On this basis, the public opinion analysis and prediction subsystem calculates the text information weight and calculates the importance of the text information for subsequent data mining processes.

[0065] The keyword frequency analysis (TF-IDF) algorithm is the basis for online public opinion analysis and prediction. It is used to evaluate the importance of a word to a document set or a document in a corpus. The importance of a word increases in direct proportion to the number of times it appears in a document, but at the same time it decreases in inverse proportion to the frequency of its appearance in the corpus. That is, if a word appears frequently in an article and rarely appears in other articles, it is considered that this word or phrase has good category differentiation ability and is suitable for classification.

[0066] The keyword frequency analysis is performed based on statistical principles, where the term frequency TF (Term Frequency) represents the number of times the word appears in the text, and the importance of the word is proportional to it. The frequency of word w in document d is determined by the following formula:

[0067]

[0068] count(d,w) is the number of times word w appears in document d; count(d,*) is the total number of words in document d;

[0069] Inverse Document Frequency (IDF) represents the frequency of the word appearing in all the texts in the text set, and the importance of the word is inversely proportional to it. If a word appears in many documents, it can be determined that the importance of the word is not high; otherwise, it is determined that the importance of the word is very high, mainly considering the importance of the word. The larger the number of documents and the fewer documents a word appears in, the larger the IDF value is, and the more important the word is. The IDF calculation method for word w is as follows:

[0070]

[0071] N: The total number of documents in the corpus.

[0072] N(w): How many documents does word w appear in.

[0073] TF-IDF combines the two to weight the importance of words. The larger the TF-IDF value of a word, the more important it is. The TF-IDF calculation formula for word w is as follows:

[0074] TF-IDF(w)=TF(d,w)*IDF(w)

[0075] Furthermore, the public opinion analysis and prediction subsystem performs topic detection and tracking. From the text feature vectors of numerous online public opinion information, through text clustering and classification, it digs out the online public opinion topic centers contained therein, tracks and reports on them, and records the entire process of topic germination, development, and extinction.

[0076] Specifically, the public opinion analysis and prediction subsystem adopts a single-pass clustering algorithm, namely the SinglePass clustering algorithm. First, a preset threshold is determined, and the texts in the text set are processed sequentially in the order in which they are obtained. Initially, the first text is used as a seed to create the first topic cluster. The texts input in sequence are then compared for similarity with each discovered topic cluster. If the similarity is greater than the preset threshold, the text is classified into this topic cluster. If it is not greater than the preset threshold, a new topic category is created using the text as a seed.

[0077] It should be noted that the preset threshold is between [0,1] and is determined according to actual scenario requirements or experience values. The higher the threshold is set, the smaller the cluster granularity (the fewer texts in the topic) and the more topic clusters there are; on the contrary, the lower the threshold is set, the larger the cluster granularity (the more texts in the topic) and the fewer topic clusters there are. The texts in the text set are processed in the order of their acquisition time. Initially, the first topic cluster is created with the first text as the seed. The texts entered in sequence are compared with each existing topic cluster for similarity. If the similarity is greater than the preset threshold, the text is classified into the topic cluster. If it is not greater than the judgment threshold, the text is used as the seed to create a new topic cluster.

[0078] Specifically, the public opinion analysis and prediction subsystem uses the first document as a seed to establish a topic cluster; vectorizes the document; calculates the similarity between the document and all existing topic clusters; determines the existing topic cluster with the maximum similarity to the document; if the maximum similarity value is greater than a preset threshold, the document is added to the topic cluster corresponding to the maximum similarity, and the feature center of the topic cluster is updated; if the maximum similarity value is less than the preset threshold, the document does not belong to any existing topic cluster, a new topic cluster is created, and the current text is assigned to the newly created topic cluster, and the vector of the text is used as the center vector of the topic cluster; clustering is completed, waiting for the next document to enter, and the above clustering steps are repeated;

[0079] Since the public opinion news data stream itself has time characteristics, the news topics are input in order of publication time, and the newly collected documents are sorted according to the update time. The earlier ones are clustered first, and the newer ones are clustered later.

[0080] In order to cope with the problem of a sharp increase in the number of topics due to the increase in time, the timeliness of network hot spots is considered, and the topic clusters are effectively selected based on the periodic analysis of the popularity.

[0081] In order to deal with the problem of low efficiency of comparing documents one by one as the number of documents increases, a "topic feature center" is added. During the clustering process, the feature center of the topic cluster is formed and continuously updated. The feature center represents a summary of the core content of all documents in the current topic cluster. When a new document is collected for comparison calculation, it only needs to be compared with the feature center of the topic cluster to find the similarity between the two, and then it can be known whether the text information belongs to the current topic cluster. In this way, the number of document comparisons can be effectively reduced, thereby improving the efficiency of the algorithm.

[0082] Furthermore, the public opinion analysis and prediction subsystem analyzes the discovered topic clusters, matches and filters them with the sensitive keyword database in the database, identifies the sensitive topic information contained therein, and issues corresponding warnings.

[0083] Specifically, the main function of sensitive topic identification is to analyze the discovered topic clusters, match and filter them with the sensitive keyword database in the database, identify the sensitive topic information contained therein, and issue corresponding warnings. The calculation of the sensitivity of the topic is mainly divided into two parts, one is the title of the topic in the text information, and the other is the summary of the topic in the text information, and the two have different weights. According to the characteristics of online public opinion information, its title is the essence of the main information to be conveyed, so the title of the topic should have a higher weight, and the summary of the topic has a lower weight. Compare the sensitivity with the preset sensitivity threshold. If it is greater than the preset sensitivity threshold, it belongs to a sensitive topic. If it is less than the preset sensitivity threshold, it belongs to a general topic. Among them, the preset sensitivity threshold can be determined according to the actual scenario requirements or experience values.

[0084] See the sensitive topic identification algorithm process for details. Figure 5 The weight of each part of the text information can be determined according to the actual scenario requirements or experience. The title of the text information should have a higher weight, and the summary has a lower weight. Assume that the weight of the text information title is set to 0.6 and the weight of the summary is set to 0.4. The calculation formula of sensitivity is as follows:

[0085]

[0086] S is the sensitivity of the topic, nt is the number of sensitive words in the title, Nt is the total number of words in the title, ns is the number of sensitive words in the summary, and Ns is the total number of words in the summary. Finally, the sensitivity S is compared with the preset sensitivity threshold. If it is greater than the preset sensitivity threshold, it belongs to a sensitive topic. If it is less than the threshold, it belongs to a general topic.

[0087] The public opinion data center is used to store and manage the analysis and prediction data obtained from the public opinion analysis and prediction subsystem, including the online public opinion information database, the online public opinion analysis database, the system management database and the sensitive database.

[0088] The network public opinion information database stores and manages the data collected by the public opinion information collection subsystem, and provides it to the public opinion data preprocessing subsystem for processing. In view of the massive scale of data, geometric growth rate, and open and heterogeneous data types, the traditional relational database system cannot meet the requirements of high speed, high efficiency and high reliability. Therefore, a new type of distributed file system database is used to store and manage data.

[0089] The network public opinion analysis database stores and manages the analysis and prediction data obtained by the public opinion analysis and prediction subsystem, provides it to the application support platform for processing, and displays it on the system user interface of the public opinion application management subsystem. Its data is mainly structured data that has been screened and processed, with a fast growth rate. At the same time, it needs to meet a large number of content relationship matching operations, so a traditional relational database is used to store and manage data.

[0090] System management database, used to store various system configuration information, such as targeted collection of website seed URLs and corresponding filtering rules;

[0091] The sensitive word database is used to store various predefined sensitive keywords. Sensitive keywords are classified according to scale, nature, etc. The public opinion data center also includes a user database, which is used to store various user-defined keyword libraries.

[0092] The application support platform is used to support various business application systems and provide basic component functions to business application systems, providing users with transparent underlying support technologies and rich common components that enable them to quickly build applications.

[0093] The public opinion application management subsystem is used to provide an interface for the interaction between the city-wide social governance dynamic event propagation prediction system and users. It is a window for the processing and display of various business subsystems of the system, and users can use it to fully monitor online public opinion. The public opinion application management subsystem shows users various topics that currently exist on the Internet, and shows the entire process of their birth, development, and demise. It displays basic information such as source, time, and location through various graphical interfaces, and displays fluctuation trends in dimensions such as the number of reports and the number of comments.

[0094] The basic platform includes basic system software such as operating systems, computer network systems, servers, storage systems, database systems and other infrastructure and equipment.

[0095] Embodiment 1 provides a system for predicting the spread of dynamic events in urban social governance. Through automated network information collection and acquisition, it can timely identify early signs of the spread of urban social governance events and capture the initial stage of events. At the same time, with the help of the growth rate of topic information, it can predict the spread scope, influence and possible evolution path of events, help relevant departments and decision makers quickly understand the development trend of events, formulate scientific and effective response strategies and measures, and take appropriate measures in a timely manner to avoid the expansion and escalation of events.

[0096] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0097] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A system for predicting the spread of dynamic events in urban social governance, characterized in that: The system includes: a public opinion information collection subsystem, a public opinion data center, a public opinion data preprocessing subsystem, and a public opinion analysis and prediction subsystem; The public opinion information collection subsystem is used to carry out targeted collection of website information in designated areas and fields of concern to users according to the user's configuration definition; the collected public opinion information is stored in the public opinion data center in the form of web page text to form a network public opinion information database; The public opinion data preprocessing subsystem is used to perform coding recognition and text extraction processing on the public opinion information in the network public opinion information database, obtain the preprocessed public opinion information and transmit it to the public opinion analysis and prediction subsystem; The public opinion analysis and prediction subsystem is used to convert the text information in the preprocessed public opinion information into a text feature vector, perform keyword frequency analysis, and determine the keywords in the text information; based on the keywords in the text information, perform topic detection and tracking on the text feature vector, and through text clustering and classification, mine and determine topic clusters, track and report on them, and record the entire process of topic germination, development, and extinction; based on the keywords in the text information, match and filter each topic cluster with sensitive keywords, identify and determine sensitive topics, and issue warnings.

2. The system according to claim 1, characterized in that The public opinion information collection subsystem is specifically used to carry out targeted collection of public opinion information data of various public opinion carriers according to the user's configuration definition, automatically collect Internet web page information in various encoding formats, and automatically collect the information of each collection website by encoding and identifying based on the collection rules configured by each collection website site, and perform deduplication and filtering according to the site uniform resource locator URL; the collected public opinion information is stored in the public opinion data center in the form of web page text to form a network public opinion information database.

3. The system according to claim 1, characterized in that The public opinion data preprocessing subsystem is specifically used to encode and identify the public opinion information in the network public opinion information database, remove web page noise, extract the text content and perform Chinese word segmentation, generate an automatic summary, and obtain the preprocessed public opinion information.

4. The system according to claim 1, characterized in that The public opinion analysis and prediction subsystem is specifically used to convert the text information in the preprocessed public opinion information into a text feature vector; calculate the TF-IDF weight of the keyword frequency analysis of the text information; and determine the keywords in the text information according to the TF-IDF weight of the text information.

5. The system according to claim 4, characterized in that The public opinion analysis and prediction subsystem is specifically used to calculate the TF-IDF weight of text information through the following formula: TF-IDF(w)=TF(d,w)*IDF(w) Where TF(d,w) is the frequency of word w in the text information appearing in document d, count(d,w) is the number of times word w in the text information appears in document d, count(d,*) is the total number of words in document d, IDF(w) is the inverse document frequency of word w in the text information, N is the total number of documents in the corpus, N(w) is the number of documents in which word w appears in the document, and TF-IDF(w) is the TF-IDF weight of word w in the text information.

6. The system according to claim 1, characterized in that The public opinion analysis and prediction subsystem is specifically used to process the text in the text feature vector in the order of acquisition time based on the keywords in the text information, use the first text as a seed to create a first topic cluster, and sequentially determine whether the maximum similarity between the remaining texts and the existing topic clusters is greater than a preset threshold; if greater, the text is classified into the topic cluster corresponding to the maximum similarity; If not, the text is used as a seed to create a new topic cluster; Follow up reports on each topic cluster, recording the entire process of topic germination, development, and extinction.

7. The system according to claim 1, characterized in that The public opinion analysis and prediction subsystem is specifically used to process the text in the text feature vector in the order of acquisition time based on the keywords in the text information, use the first text as a seed to create a first topic cluster, and determine the feature center of the topic cluster, and sequentially determine whether the maximum similarity between the remaining texts and the feature center of the existing topic cluster is greater than a preset threshold; if greater, classify the text into the topic cluster corresponding to the maximum similarity, and update the feature center of the topic cluster; If it is not greater than, the text is used as a seed to create a new topic cluster, and the feature center of the new topic cluster is determined; Follow up reports on each topic cluster and record the entire process of topic germination, development, and extinction.

8. The system according to claim 1, characterized in that The public opinion analysis and prediction subsystem is specifically used to match and filter each topic cluster with sensitive keywords based on the keywords in the text information, determine the sensitivity of the topic according to the weight of each part of the text information; judge whether the sensitivity of the topic is greater than a preset sensitivity threshold; if greater, determine it as a sensitive topic and issue a warning; If it is less than, it is determined to be a general topic.

9. The system according to claim 8, characterized in that The public opinion analysis and prediction subsystem is specifically used to match and filter each topic cluster with sensitive keywords based on the keywords in the text information, and determine the topic sensitivity according to the weight of each part of the text information through the following formula: Among them, S is the topic sensitivity, nt is the number of sensitive words contained in the text information title, Nt is the total number of words in the text information title, ns is the number of sensitive words contained in the text information summary, Ns is the total number of words in the text information summary, a is the text information title weight, and b is the text information summary weight.

10. The system according to claim 1, characterized in that The system also includes a public opinion application management subsystem; The public opinion application management subsystem is used to provide an interactive interface between the system and the user, display the prediction results of the public opinion analysis and prediction subsystem, and display the entire process of the germination, development, and extinction of various topics as well as the basic information of various topics; The public opinion data center includes an online public opinion information database, an online public opinion analysis database, a system management database, and a sensitive word database; among them, the online public opinion information database is used to store and manage the collected data of the public opinion information collection subsystem using a distributed file system database; the online public opinion analysis database is used to store and manage the prediction result data of the public opinion analysis and prediction subsystem using a relational database, and provide it to the public opinion application management subsystem for display on the interactive interface; the system management database is used to store various configuration information of the system; the sensitive word database is used to store various predefined sensitive keywords, and the sensitive keywords are graded according to scale and nature.

Images