Community media public opinion real-time monitoring and analysis system

By designing a real-time monitoring and analysis system for public opinion in the community media, and using technical means such as multi-source collection, incremental collection, precise cleaning and advanced model analysis, the problems of real-time collection, accurate processing and in-depth analysis of community public opinion are solved, real-time and accurate collection and analysis of community public opinion are achieved, providing community managers with comprehensive and accurate public opinion information, and improving the level of community management.

CN120216680AInactive Publication Date: 2025-06-27SHANGHAI YUESHAN CULTURE MEDIA GRP CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510696886.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-28
Publication Date
2025-06-27
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

It is difficult for existing technology to efficiently and comprehensively collect, accurately process and in-depth analyze valuable public opinion information from community media, resulting in negative public opinion not being detected in a timely manner, affecting the community image and the quality of life of residents.

Method used

Design a real-time monitoring and analysis system for public opinion in the community media, including multi-source collection unit, real-time update unit, data cleaning unit, text word segmentation unit, emotion analysis unit, theme classification unit, trend analysis unit, association analysis unit, visual display unit and early warning unit. Through technical means such as adaptive algorithms, incremental acquisition, precise cleaning, advanced model analysis and multi-condition early warning, real-time collection, precise processing and in-depth analysis of community public opinion.

Benefits of technology

Real-time and accurate collection and analysis of community public opinion is achieved, comprehensive and accurate public opinion information is provided, and community managers can make decisions in a timely and scientific manner and improve community management level.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120216680A_ABST
    Figure CN120216680A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of public opinion monitoring, and particularly discloses a community media public opinion real-time monitoring and analysis system, which is characterized in that a multi-source acquisition unit acquires data for different community media platforms by utilizing a web crawler; the real-time updating unit adjusts a collection interval according to the activity degree of community information and collects newly-added data; the data cleaning unit cleans the data; the text word segmentation unit performs word segmentation on the cleaned text by using a BiLSTM-CRF model; the sentiment analysis unit judges the text sentiment tendency after word segmentation; the subject classification unit classifies public opinions according to the text features; the trend analysis unit analyzes a public opinion trend; the association analysis unit mines a public opinion association relationship; the visual display unit displays the analysis result through a chart; the early warning unit carries out early warning through short messages and mails. According to the invention, real-time acquisition, precise processing, deep analysis, effective display and early warning of community media public opinions are realized, comprehensive and accurate public opinion information is provided for community managers, the community managers are assisted to make decisions timely and scientifically, and the community management level is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of public opinion monitoring, and specifically refers to a real-time monitoring and analysis system for community media public opinion. Background Art

[0002] In the current wave of digital and information technology, community media, as a key platform for residents' information interaction and emotional expression, has become increasingly important. Various media channels such as community-exclusive forums, social media groups, and official website bulletin boards carry the rapid spread and interaction of a vast amount of information every day.

[0003] On the one hand, community residents use these media platforms to share daily life details, feedback on community issues, discuss public affairs, etc. The amount of information generated is huge and the update speed is extremely fast. For example, on a community forum, residents may quickly initiate discussions on newly introduced community management regulations, and a large number of related posts and comments will be generated in a short time; in social media groups, notifications and discussions about community activities can accumulate hundreds of chat records within a few hours.

[0004] On the other hand, community managers are faced with severe challenges in public opinion monitoring and analysis. Existing technologies are difficult to efficiently and comprehensively collect valuable public opinion information in real time from such complex multi-source data. Traditional collection methods are often limited to a single platform or rely on manual screening, which is not only inefficient but also extremely likely to miss important information, resulting in the rapid fermentation of negative public opinion without being detected, causing negative impacts on the community image and the quality of residents' lives.

[0005] In the data processing link, in the face of data in community media that contains a large amount of interference information such as garbled characters, residual HTML tags, duplicate data, and irrelevant advertisements, existing cleaning technologies are difficult to process accurately and efficiently, resulting in poor quality of the data basis for subsequent analysis. Moreover, for special language phenomena such as new words and internet terms that frequently appear in community texts, conventional text segmentation technologies cannot accurately segment them, seriously affecting the understanding and analysis of public opinion content.

[0006] In terms of public opinion recognition and analysis, existing systems lack effective models optimized for the characteristics of community public opinion. The judgment of the emotional tendency and theme classification of community public opinion is not accurate enough, it is difficult to accurately extract key information from complex texts, and it is even more impossible to deeply explore the internal correlation relationships between different public opinion themes and between public opinion and community events, and cannot provide comprehensive, in-depth, and forward-looking decision-making support for community managers.

[0007] In terms of result presentation and early warning, the display methods provided by the existing technologies are not intuitive and flexible enough to meet the needs of community managers to quickly and clearly understand the overall situation of public opinion. At the same time, the early warning mechanisms are often set simply and have fixed thresholds, making it difficult to issue timely and accurate early warnings according to the diversity and dynamic characteristics of community public opinion, resulting in community managers being unable to intervene and handle in a timely manner at the initial stage of public opinion.

[0008] Therefore, a real-time monitoring and analysis system for community media public opinion has become an urgent problem to be solved. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to provide a real-time monitoring and analysis system for community media public opinion, which can realize real-time collection, precise processing, in-depth analysis, effective display and early warning of community media public opinion, provide comprehensive and accurate public opinion information for community managers, assist them in making decisions in a timely and scientific manner, and improve the level of community management.

[0010] To solve the above technical problem, the technical solution provided by the present invention is: a real-time monitoring and analysis system for community media public opinion, including:

[0011] Data acquisition module: including a multi-source acquisition unit and a real-time update unit. The multi-source acquisition unit uses web crawlers to collect data for different community media platforms with an adaptive algorithm; the real-time update unit adjusts the acquisition interval according to the activity of community information and acquires new data by incremental acquisition;

[0012] Data preprocessing module: including a data cleaning unit and a text tokenization unit. The data cleaning unit cleans data through string matching, regular expressions and a keyword blacklist library; the text tokenization unit tokenizes the cleaned text with a BiLSTM-CRF model;

[0013] Public opinion recognition module: including a sentiment analysis unit and a topic classification unit. The sentiment analysis unit judges the sentiment tendency of the tokenized text with CNN combined with pre-trained word vectors; the topic classification unit classifies public opinion according to text features with an SVM algorithm;

[0014] Public opinion analysis module: including a trend analysis unit and an association analysis unit. The trend analysis unit analyzes the public opinion trend with the moving average method and the exponential smoothing method; the association analysis unit mines the public opinion association relationship with the Apriori algorithm;

[0015] Result display and early warning module: including a visualization display unit and an early warning unit. The visualization display unit displays the analysis results in charts with the Echarts library; the early warning unit issues early warnings by text message and email when a specific public opinion reaches the threshold.

[0016] Further, the adaptive algorithm realizes automatic platform type identification and acquisition strategy adjustment by combining pattern matching and rule reasoning based on the feature recognition of data structures and access rules on different platforms.

[0017] Further, the real-time update unit sets regular acquisition tasks, flexibly adjusts the acquisition time interval according to the updated activity of community information, shortens it to 5 minutes during the information active period, extends it to 30 minutes during the inactive period, and adopts an incremental acquisition method, only obtaining the newly added data after the last acquisition time point each time.

[0018] Further, the data cleaning unit uses a string matching algorithm to identify and delete duplicate data, uses regular expression matching rules to remove the remaining HTML tags and invalid characters, filters out the interfering information containing blacklist keywords by establishing a keyword blacklist library, and cleans the collected data to remove duplicate data, invalid characters, and interfering information irrelevant to community public opinion.

[0019] Further, the keyword blacklist library is constructed based on community management experience, common advertising vocabulary, and the characteristics of past irrelevant information, and is dynamically updated.

[0020] Further, the text tokenization unit adopts the BiLSTM-CRF model, trains and optimizes it according to the characteristics of community media texts, and performs fine-tuning using specific corpora in the community media field during the training process, splitting the text content in the cleaned data into individual words or phrases.

[0021] Further, the sentiment analysis unit uses a CNN model combined with pre-trained word vectors to train the community public opinion text data with labeled sentiment tendencies, extracts and classifies text features through convolutional layers, pooling layers, and fully connected layers, and judges the sentiment tendency of the tokenized text, which is divided into positive, negative, and neutral;

[0022] The topic classification unit uses the SVM algorithm to construct a classifier based on the topic characteristics of community public opinion, extracts keywords, word frequencies, and part-of-speech in the text, inputs these feature vectors into the SVM classifier for training and classification, and divides the public opinion information into different topic categories.

[0023] Further, the trend analysis unit uses the moving average method and the exponential smoothing method, dynamically adjusts the smoothing coefficient according to the volatility characteristics of public opinion data, processes the public opinion data, analyzes the development trends of public opinion with different topics and different sentiment tendencies in terms of time series, and draws a public opinion trend curve;

[0024] The association analysis unit uses the Apriori algorithm to mine frequent item sets and learn association rules from public opinion data, and mines the association relationships between different public opinion topics and between public opinion and community events;

[0025] Furthermore, the visualization display unit uses the Echarts visualization library to design a visualization interface and display the public opinion analysis results in the form of charts, including a pie chart of public opinion theme distribution, a bar chart of sentiment tendency proportion, and a line chart of public opinion trend.

[0026] The early warning unit sets public opinion early warning rules. When public opinion with a specific theme and specific sentiment tendency reaches a set threshold, an early warning mechanism is automatically triggered by text message or email to send early warning information to the community manager.

[0027] Furthermore, the early warning rules setting of the early warning unit supports multi-condition combinations, including combinations of public opinion theme, sentiment tendency, change range of public opinion quantity, and time span.

[0028] The advantages of the present invention compared with the prior art are as follows:

[0029] Through the real-time update unit of the data acquisition module, the present invention can flexibly adjust the acquisition interval according to the community information activity, adopt an incremental acquisition method, ensure timely acquisition of the latest public opinion information, and enable the community manager to master the public opinion dynamics in the first time.

[0030] The precise cleaning and word segmentation of the data preprocessing module, as well as the advanced model adopted by the public opinion recognition module, can effectively improve the accuracy of public opinion recognition and classification, and provide a reliable data basis for subsequent analysis.

[0031] The trend analysis and correlation analysis functions of the public opinion analysis module can deeply explore the potential information behind the public opinion and provide comprehensive and scientific decision-making support for the community manager.

[0032] The visualization display unit intuitively presents the public opinion analysis results in the form of charts, facilitating the community manager to quickly understand and grasp the public opinion situation.

[0033] The multi-condition combination early warning rules setting of the early warning unit can timely discover potential public opinion risks and notify the community manager in time by text message, email, etc., enabling it to take corresponding measures in time. Description of the Drawings

[0034] Figure 1 is the system block diagram of a community media public opinion real-time monitoring and analysis system of the present invention.

[0035] Figure 2 is the working flow chart of a community media public opinion real-time monitoring and analysis system of the present invention. Detailed Embodiments

[0036] Various exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that: Unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present invention.

[0037] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way a limitation on the present invention or its application or use.

[0038] Techniques, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered as part of the specification.

[0039] In all examples shown and discussed herein, any specific values should be construed as merely exemplary and not as a limitation. Thus, other examples of the exemplary embodiments may have different values.

[0040] The following further details the real-time monitoring and analysis system for community media public opinion of the present invention in conjunction with the accompanying drawings.

[0041] Combined with the attached Figure 1 - Figure 2 , the present invention is introduced in detail.

[0042] A real-time monitoring and analysis system for community media public opinion mainly consists of a data acquisition module, a data preprocessing module, a public opinion recognition module, a public opinion analysis module, and a result display and warning module.

[0043] The data acquisition module includes a multi-source acquisition unit and a real-time update unit.

[0044] Multi-source acquisition unit: With the help of web crawler technology, data acquisition work is carried out for various community media platforms such as community-exclusive forums, social media groups, and official website bulletin boards. An adaptive acquisition algorithm is adopted, which is based on the feature recognition of the data structures and access rules of different platforms, and uses a combination of pattern matching and rule reasoning to automatically identify the platform type and adjust the acquisition strategy accordingly.

[0045] Data feature analysis and pattern extraction:

[0046] For community-exclusive forums, analyze the page HTML structure to determine the HTML tags where information such as post titles, contents, release times, and comments are located, as well as their hierarchical relationships and attribute characteristics. By analyzing and summarizing the page structures of multiple different forums, a general pattern of the forum platform data structure is extracted.

[0047] For social media groups, study the data format returned by their open interfaces. For example, in JSON-formatted data, determine the field names, data types, and hierarchical relationships of information such as in-group chat records and member statements. For instance, chat records may be in the content sub-field under the message field, and member information may be in relevant sub-fields under the user field.

[0048] For the official website bulletin board, analyze the display form of the bulletin content, such as the layout and format features of information like the bulletin title, release time, and body text, and extract the pattern of the bulletin board data structure.

[0049] Understand the restrictions on data access for different platforms, such as request frequency limits, request header requirements, authentication methods, etc. For example, some forums may limit the number of requests per minute to not exceed a certain amount, and the interfaces of social media groups may require a specific authentication token to be carried in the request header. By analyzing and summarizing the access rules of multiple platforms, extract the patterns of access rules for different platforms.

[0050] Pattern matching implementation:

[0051] Build a pattern matching library: Organize the patterns of the platform data structure and access rules extracted above into a pattern matching library. The pattern matching library can be stored using data structures (such as dictionaries, lists, etc.), with each platform corresponding to a set of pattern information. For example, for the forum platform, the pattern matching library may contain a dictionary with the key "forum" and the value being a sub-dictionary containing the data structure pattern and the access rule pattern.

[0052] Pattern matching algorithm: When collecting data, use the pattern matching algorithm to match the target platform. For a new platform, first obtain some of its data (such as a page HTML fragment or a partial data returned by an interface), and then compare this data with the patterns in the pattern matching library one by one. When comparing, methods such as string matching and regular expression matching can be used to determine whether the data structure and access rules match the pattern of a certain platform. If the match is successful, determine the type of the platform.

[0053] Rule inference implementation:

[0054] Build a rule inference engine: According to the characteristics of different platforms and data collection requirements, build a rule inference engine. The rule inference engine can be implemented using a rule-based system (such as a production system), and the rules can be represented in the form of condition-action. For example, the rule "If the platform type is a forum and the data collection volume is large, then adopt a multi-threaded concurrent collection method to improve the collection efficiency".

[0055] Inference process: After determining the platform type, the rule inference engine matches the corresponding rules from the rule library based on the platform type and the current collection situation (such as the collection volume, collection time interval, etc.), and executes the action part of the rules, that is, adjusts the collection strategy. For example, if the platform type is a social media group and the current collection frequency exceeds the platform limit, the inference engine will adjust the collection time interval according to the rules to reduce the collection frequency.

[0056] Automatic adjustment of the collection strategy:

[0057] Collection strategy adjustment mechanism: According to the results of rule inference, automatically adjust the relevant parameters and operations of the data collection module. For example, adjust the request headers, request frequency, collection time interval, number of collection threads, etc. of the web crawler. If the inference result is that multi-threaded concurrent collection is required, start multiple collection threads and perform data collection according to the adjusted collection strategy.

[0058] Dynamic monitoring and optimization: During the data collection process, monitor the collection situation and platform feedback information in real time, and dynamically adjust the collection strategy according to the actual situation. If it is found that the quality of the collected data is not good or the platform has an abnormal response, re-perform pattern matching and rule inference, and adjust the collection strategy to ensure the accuracy and stability of data collection.

[0059] Real-time update unit: Set a regular collection task, and flexibly adjust the collection time interval according to the updated activity of the community information. During the information active period (such as 7-10 pm), shorten the collection interval to 5 minutes; during the inactive period, extend the collection interval to 30 minutes. At the same time, adopt an incremental collection method, and only obtain the newly added data after the last collection time point each time, so as to reduce data redundancy and collection workload.

[0060] The data preprocessing module includes a data cleaning unit and a text tokenization unit.

[0061] Data cleaning unit: Clean the collected data. Use string matching algorithms to identify and delete duplicate data; use regular expression matching rules to remove HTML tag residues and invalid characters; filter out interference information containing blacklist keywords, such as advertising links and meaningless notifications automatically pushed by the system, through the establishment of a keyword blacklist library. This keyword blacklist library is constructed based on community management experience, common advertising vocabulary, and the characteristics of past irrelevant information, and can be updated dynamically.

[0062] Text Word Segmentation Unit: Adopts a Bidirectional Long Short-Term Memory Network combined with a Conditional Random Field (BiLSTM-CRF) model, and is trained and optimized according to the characteristics of community media texts. During the training process, specific corpora in the community media field are used for fine-tuning, and the text content in the cleaned data is segmented into individual words or phrases to effectively handle special situations such as new words and internet buzzwords in community texts. For example, the correct segmentation of combined words such as "community group buying" and "garbage classification points".

[0063] The Public Opinion Recognition Module includes a Sentiment Analysis Unit and a Topic Classification Unit.

[0064] Sentiment Analysis Unit: Adopts a Convolutional Neural Network (CNN) model combined with pre-trained word vectors (such as Word2Vec), and is trained on a large amount of community public opinion text data with labeled sentiment tendencies. The model extracts and classifies text features through convolutional layers, pooling layers, and fully connected layers, and judges the sentiment tendency of the segmented text, which is divided into positive, negative, and neutral. For example, for texts such as "The newly installed fitness equipment in the community is great, enriching our amateur life", it can be accurately identified as positive sentiment; for "The garbage at the entrance of the community is often not cleaned in time, and the environment is too bad", it is judged as negative sentiment.

[0065] Topic Classification Unit: Utilizes the Support Vector Machine (SVM) algorithm to construct a classifier based on the topic characteristics of community public opinion. By extracting features such as keywords, word frequencies, and part-of-speech in the text, these feature vectors are input into the SVM classifier for training and classification, and the public opinion information is divided into different topic categories, such as community environment, property service, cultural activities, facility construction, etc. When inputting a public opinion message "The literary and art evening organized by the community is very wonderful, and I hope it will be held more in the future", the classifier can accurately classify it into the "cultural activities" topic.

[0066] The Public Opinion Analysis Module includes a Trend Analysis Unit and a Correlation Analysis Unit.

[0067] Trend Analysis Unit: Adopts the moving average method and the exponential smoothing method, dynamically adjusts the smoothing coefficient according to the fluctuation characteristics of public opinion data, and processes the public opinion data. Taking the time series as the dimension, analyzes the development trends of public opinions with different topics and different sentiment tendencies, and draws the public opinion trend curve. For example, by analyzing the quantity change curve of negative public opinions related to the community environment within a period of time, visually displays the fluctuation of such public opinions, and predicts the future development trend, providing a basis for community managers to formulate countermeasures in advance.

[0068] Association analysis unit: Using the Apriori algorithm, it conducts frequent itemset mining and association rule learning on public opinion data to mine the association relationships between different public opinion themes and between public opinion and community events. For example, it discovers a strong association between the public opinion of "inadequate and untimely elevator maintenance" under the theme of "property service" and the public opinion of "elevator aging" under the theme of "facility construction", and prompts community managers to consider solutions to elevator-related problems as a whole.

[0069] The result display and warning module includes a visualization display unit and a warning unit.

[0070] Visualization display unit: Using the Echarts visualization library, it designs a visualization interface to display the results of public opinion analysis in the form of charts, including a pie chart of the distribution of public opinion themes, a bar chart of the proportion of sentiment tendencies, a line chart of public opinion trends, etc. Through these intuitive charts, it is convenient for community managers to quickly understand the overall situation of public opinion. For example, the pie chart clearly presents the public opinion themes with higher attention in the current community.

[0071] Warning unit: It sets public opinion warning rules. When public opinion with a specific theme and a specific sentiment tendency reaches the set threshold, it automatically triggers a warning mechanism to send warning messages to community managers via text messages, emails, etc. The setting of warning rules supports multi-condition combinations, including combinations of conditions such as public opinion themes, sentiment tendencies, the change range of the number of public opinions, and time spans, ensuring that community managers can intervene in a timely manner at the initial stage of public opinion.

[0072] The specific implementation process of a real-time monitoring and analysis system for community media public opinion of the present invention is as follows:

[0073] After the system is started, the data collection module collects community media information in real time according to the set rules and parameters. The multi-source collection unit automatically identifies the platform type and adjusts the collection strategy according to the adaptive collection algorithm; the real-time update unit adjusts the collection interval according to the activity of community information and uses the incremental collection method to obtain new data.

[0074] The collected data enters the data preprocessing module. The data cleaning unit cleans the data to remove duplicate data, invalid characters, and interference information; the text tokenization unit uses the BiLSTM-CRF model to tokenize the cleaned data.

[0075] The preprocessed data is input into the public opinion recognition module. The sentiment analysis unit and the theme classification unit respectively judge the sentiment tendency and classify the theme of the text. Then, the trend analysis unit and the association analysis unit of the public opinion analysis module analyze the recognition results, draw the public opinion trend curve, and mine the public opinion association relationships.

[0076] The results of public opinion analysis are displayed in the form of charts on the visualization interface through the visualization display unit for community managers to view. At the same time, the warning unit monitors public opinion data in real time. When a specific public opinion reaches the set threshold, the warning mechanism is automatically triggered, and warning messages are sent to community managers via text messages, emails, etc.

[0077] The system is maintained and optimized regularly, including updating the keyword blacklist library, adjusting model parameters, checking the system operation status, etc., to ensure the stable operation of the system and the accuracy of the analysis results.

[0078] The above describes the present invention and its implementation manners. Such a description is not restrictive. What is shown in the drawings is only one of the implementation manners of the present invention, and the actual structure is not limited thereto. In general, if those of ordinary skill in the art are inspired by it and design similar structural manners and embodiments to this technical solution without creative efforts without departing from the purpose of the present invention, they shall fall within the protection scope of the present invention.

Claims

1. A real-time monitoring and analysis system for community media public opinion, characterized in that, Including: Data acquisition module: including a multi-source acquisition unit and a real-time update unit. The multi-source acquisition unit uses web crawlers and an adaptive algorithm to collect data for different community media platforms; The real-time update unit adjusts the acquisition interval according to the activity of community information and uses incremental acquisition to add new data; Data preprocessing module: including a data cleaning unit and a text tokenization unit. The data cleaning unit cleans data through string matching, regular expressions, and a keyword blacklist library; the text tokenization unit tokenizes the cleaned text using a BiLSTM-CRF model; Public opinion recognition module: including a sentiment analysis unit and a topic classification unit. The sentiment analysis unit uses a CNN combined with pre-trained word vectors to judge the sentiment tendency of the tokenized text; the topic classification unit classifies public opinion according to text features using an SVM algorithm; Public opinion analysis module: including a trend analysis unit and an association analysis unit. The trend analysis unit analyzes the public opinion trend using the moving average method and the exponential smoothing method; The association analysis unit mines the association relationship of public opinion using the Apriori algorithm; Result display and warning module: including a visualization display unit and a warning unit. The visualization display unit uses the Echarts library to display the analysis results in charts; The warning unit issues warnings via text messages and emails when the public opinion reaches the threshold.

2. The real-time monitoring and analysis system for community media public opinion according to claim 1, wherein: The adaptive algorithm is based on the feature recognition of the data structures and access rules of different platforms, and realizes the automatic recognition of platform types and the adjustment of acquisition strategies by combining pattern matching and rule reasoning.

3. A real-time monitoring and analysis system for community media public opinion according to claim 2, characterized in that: The real-time update unit sets a regular acquisition task, flexibly adjusts the acquisition time interval according to the update activity of community information, shortens it to 5 minutes during the information active period, extends it to 30 minutes during the inactive period, and uses the incremental acquisition method to only obtain the new data after the last acquisition time point each time.

4. The real-time monitoring and analysis system for community media public opinion according to claim 3, characterized in that: The data cleaning unit uses a string matching algorithm to identify and delete duplicate data, uses regular expression matching rules to remove HTML tag residues and invalid characters, and filters out interference information containing blacklist keywords by establishing a keyword blacklist library, cleans the collected data, and removes duplicate data, invalid characters, and interference information irrelevant to community public opinion.

5. The real-time monitoring and analysis system for community media public opinion according to claim 4, characterized in that: The keyword blacklist library is constructed based on community management experience, common advertising vocabulary, and the characteristics of past irrelevant information, and is dynamically updated.

6. The real-time monitoring and analysis system for community media public opinion according to claim 5, characterized in that: The text tokenization unit uses a BiLSTM-CRF model, trains and optimizes it according to the characteristics of community media texts, and fine-tunes it using the corpus in the community media field during the training process, and splits the text content in the cleaned data into individual words or phrases.

7. The real-time monitoring and analysis system for community media public opinion according to claim 6, characterized in that: The sentiment analysis unit uses a CNN model combined with pre-trained word vectors to train the community public opinion text data with labeled sentiment tendency, extracts and classifies the text features through convolutional layers, pooling layers, and fully connected layers, and judges the sentiment tendency of the tokenized text, which is divided into positive, negative, and neutral; The described topic classification unit uses the SVM algorithm to construct a classifier based on the topic characteristics of community public opinion. By extracting keywords, word frequencies, and part-of-speech in the text, these feature vectors are input into the SVM classifier for training and classification, and the public opinion information is divided into different topic categories.

8. The real-time monitoring and analysis system for community media public opinion according to claim 7, characterized in that: The described trend analysis unit adopts the moving average method and the exponential smoothing method. According to the fluctuation characteristics of public opinion data, the smoothing coefficient is dynamically adjusted to process the public opinion data. Taking the time series as the dimension, it analyzes the development trends of public opinion with different topics and different sentiment tendencies, and draws the public opinion trend curve. The described association analysis unit uses the Apriori algorithm to mine frequent item sets and learn association rules from public opinion data, and discovers the association relationships between different public opinion topics and between public opinion and community events.

9. The real-time monitoring and analysis system for community media public opinion according to claim 8, characterized in that: The described visualization display unit uses the Echarts visualization library to design a visualization interface and display the public opinion analysis results in the form of charts, including the pie chart of public opinion topic distribution, the bar chart of sentiment tendency proportion, and the line chart of public opinion trend. The described warning unit sets public opinion warning rules. When the public opinion with a certain topic and sentiment tendency reaches the set threshold, the warning mechanism is automatically triggered by text message or email to send warning information to the community manager.

10. A real-time monitoring and analysis system for community media public opinion according to claim 9, characterized in that: The warning rules of the described warning unit are set by combining multiple conditions, including the combination settings of public opinion topics, sentiment tendencies, the change range of public opinion quantity, and time span.

Citation Information

Patent Citations

  • Whole-network big data public opinion monitoring system based on cloud media

    CN118349626A

  • Method and system for generating a transitory sentiment community

    US20200184345A1