Multi-source public opinion data processing method for establishing standard specification
By establishing standard and standardized multi-source public opinion data processing methods, the problems of cumbersome processing of public opinion data and difficult to understand emotional tendencies in the existing technology are solved, and accurate monitoring and early warning of public opinion events are achieved to prevent the occurrence of sudden events.
Patent Information
- Application Number
- CN202411564114.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-05
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing public opinion data is complicated to process and difficult to integrate and process, which leads to insufficient understanding of public opinion events and difficulty in accurately understanding public emotional tendencies, which can easily lead to emergencies.
Establish standard and standardized multi-source public opinion data processing methods, and obtain and integrate multi-channel public opinion data by building data acquisition channels, establishing public opinion databases, conducting data monitoring and emotional analysis, obtaining and integrating multi-channel public opinion data, and generating analysis reports to understand public emotional tendencies and prevent emergencies.
It has achieved understanding of the emotional tendencies of public opinion events, timely monitoring and early warning, avoiding the occurrence of sudden events, and improving the pertinence and effectiveness of public opinion handling.
Smart Images

Figure CN119066249A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of public opinion data processing, and in particular to a method for processing public opinion data from multiple sources with a standard specification. Background Art
[0002] The existing public opinion data processing is a very tedious task, which often requires the platform to capture public opinion data from various channels. These public opinion data are large and complex, so it is difficult to integrate and process the data, resulting in an insufficient understanding of public opinion events and insufficient understanding of the public's emotional tendencies towards public opinion events. Therefore, it is easy to lead to inadequate guidance and handling of public opinion and the occurrence of sudden incidents. Summary of the invention
[0003] 1. Purpose of the invention
[0004] In view of this, the purpose of the present invention is to propose a standard and normative multi-source public opinion data processing method, through which public opinion data can be obtained from multiple channels, and the public opinion data obtained from multiple channels can be analyzed and integrated, so as to understand the public's emotional tendencies towards public opinion events, so as to facilitate subsequent artificial intervention and guidance when the negative emotional tendencies generated by public opinion events cause negative situations among the public, so as to prevent the occurrence of sudden events.
[0005] (II) Technical solution
[0006] In order to achieve the above technical objectives, the present invention provides a method for processing public opinion data from multiple sources to establish a standard specification, and the specific steps are as follows:
[0007] Public opinion data acquisition: build a public opinion data acquisition channel, preset the data collection frequency, and collect public opinion data from the public opinion data acquisition channel at the preset frequency;
[0008] Establish a public opinion database: Analyze the acquired public opinion data and build a public opinion database;
[0009] Public opinion data monitoring: Analyze public opinion data and monitor specific public opinions;
[0010] Generate public opinion analysis report: form a standardized analysis report on public opinion of a specific event.
[0011] As a further description of the above technical solution: the public opinion data acquisition channels include: internal data synchronization of unit groups, internal data sharing of cooperative enterprises, and third-party public data capture. Third-party public data includes but is not limited to: social media data (such as Weibo, WeChat, Twitter), news website data, forum data, blog data, comment platform data. The data collection tool adopts one of the following: crawler technology, API interface, data capture tool.
[0012] As a further description of the above technical solution: The specific method of establishing a public opinion database is as follows:
[0013] Public opinion data includes video data, text data, picture data, public comment data, and public likes data of public opinion events;
[0014] Identify the video data, text data and picture data in the public opinion data, and determine whether the collected video data, text data and picture data contain the same public opinion event. If it is determined that the collected video data, text data and picture data contain the same public opinion event, match, integrate and clean the video data, text data and picture data of the same public opinion event to form a public opinion data chain;
[0015] Determine whether the current public opinion data chain is associated with the historical public opinion data chain. If it is determined that the current public opinion data chain is associated with the historical public opinion data chain, merge the current public opinion data chain with the historical public opinion data chain, clean the duplicate parts, and establish a public opinion event data set;
[0016] Based on the initial time of the public opinion event, the public opinion event data set is arranged and integrated in chronological order to form a public opinion database.
[0017] As a further description of the above technical solution: the method for determining whether the collected video data, text data and picture data contain the same public opinion event is as follows:
[0018] Extract keywords from each video data, text data and picture data, wherein the keyword extraction adopts the following methods: word frequency analysis, TF-IDF, topic model;
[0019] Based on the extracted keywords, create labels for each video data, text data and image data, the labels including: the time when the public opinion event occurred, the location where the public opinion event occurred, and the description of the public opinion event;
[0020] The labels of each video data, text data and image data are compared. When the compared labels are consistent, they are determined to be the same public opinion event.
[0021] As a further description of the above technical solution: the specific steps of matching, integrating and cleaning the video data, text data and picture data of the same public opinion event to form a public opinion data chain are as follows:
[0022] Combine the video data, text data and picture data, public comment data and public likes data of the same public opinion event, and arrange them in a preset order, such as: arrange in the order of video data-picture data-text data-public comment data-public likes data or text data-picture data-video data-public comment data-public likes data;
[0023] The repeated and invalid parts in the video data, text data, image data and public comment data are eliminated to obtain a uniquely constructed public opinion data chain.
[0024] As a further description of the above technical solution: The specific method of establishing a public opinion event dataset is as follows:
[0025] Compare the label of the current public opinion data chain with the label of the historical public opinion data chain. When the compared labels are consistent, it is determined that the current public opinion data chain is associated with the historical public opinion data chain.
[0026] The current public opinion data chain is merged with the historical public opinion data chain, the duplicate parts in the merged public opinion data chain are removed, and they are rearranged in a preset order to establish a public opinion event data set.
[0027] As a further description of the above technical solution: the method of public opinion data monitoring is as follows:
[0028] Parsing the public opinion event dataset in the public opinion database:
[0029] Conduct sentiment analysis on the public opinion event data set to obtain the sentiment tendency of the public opinion events;
[0030] Monitoring specific public opinions:
[0031] Calculate the heat value of the public opinion event dataset;
[0032] Preset the heat value monitoring threshold and perform tagging monitoring on the public opinion event data set that exceeds the heat monitoring threshold;
[0033] Establish an early warning mechanism based on the public opinion event dataset with spiked monitoring.
[0034] As a further description of the above technical solution: the method of obtaining the emotional tendency of public opinion events through sentiment analysis is as follows:
[0035] Extract public comment data from the public opinion event dataset;
[0036] Analyze the sentiment tendency of each public comment through natural language processing technology, wherein the sentiment tendency includes: positive sentiment tendency, negative sentiment tendency, and neutral sentiment tendency;
[0037] Count the sentiment trends of all public comments and obtain the number of comments with positive sentiment trends, negative sentiment trends, and neutral sentiment trends respectively;
[0038] Based on the sentiment tendencies of all statistical public comments, judge the sentiment tendencies of public opinion events.
[0039] The specific judgment method is as follows:
[0040] By comparing the emotional tendencies of all public comments, when the number of comments with positive emotional tendencies is the largest, the public opinion event is judged to have a positive emotional tendency; similarly, when the number of comments with negative emotional tendencies is the largest, the public opinion event is judged to have a negative emotional tendency; when the number of comments with neutral emotional tendencies is the largest, the public opinion event is judged to have a neutral emotional tendency.
[0041] As a further description of the above technical solution: the method for calculating the heat value of the public opinion event data set is as follows:
[0042]
[0043] in, is the heat value of the public opinion event dataset, Extracted from the public opinion event dataset Number of public comments in data collection channels, Extracted from the public opinion event dataset Number of public likes in the data collection channels, , The weight of the impact of public comments and public likes on the popularity value.
[0044] The method of establishing an early warning mechanism based on the public opinion event dataset with spiked monitoring is as follows:
[0045] The heat value change collection time period is preset, and the heat value of the public opinion event data set monitored by the spike in each time period is collected, which are recorded as , ,…… ,in, For the The heat value of the public opinion event data set monitored at a certain time period;
[0046] Count the changes in heat value within each time period ,but: ,in, For the The heat value of the public opinion event data set monitored by the spiked monitoring in the time period, For the The heat value of the public opinion event data set monitored at a certain time period;
[0047] Mark the preset heat value change threshold as ,
[0048] like < , then no warning instruction is generated;
[0049] like ≥ , then an early warning instruction is generated, and based on the early warning instruction, an early warning message is sent to the platform.
[0050] As a further description of the above technical solution: the method for generating a public opinion analysis report is as follows:
[0051] Upload the public opinion standardized analysis report template;
[0052] Based on the specific public opinion being monitored, extract the data information in the public opinion event data set, write it into the corresponding public opinion standardized analysis report template, and form a standardized analysis report.
[0053] In the above technical scheme, the method for processing public opinion data from multiple sources with a standardized standard is provided by the present invention. The method can obtain public opinion data from multiple channels, and analyze and integrate the public opinion data obtained from multiple channels, so as to understand the public's emotional tendency towards public opinion events, so as to facilitate subsequent manual intervention and guidance when the negative emotional tendency generated by public opinion events causes negative situations among the public, so as to prevent the occurrence of sudden events;
[0054] This method can be used to analyze and integrate public opinion data from multiple channels, calculate the heat value of public opinion events, monitor public opinion events with higher heat values, and understand the trend of key public opinion events in real time;
[0055] This method can track the same public opinion event through multiple channels, and parallelize the same public opinion event to the existing public opinion database to form a public opinion event data set in the public opinion database. Therefore, when a public opinion event needs to be extracted, the public opinion event data set of the corresponding public opinion event can be directly retrieved from the public opinion database, so that all information of the public opinion event can be intuitively understood. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0057] Figure 1 The overall flow chart of the method for processing public opinion data from multiple sources to establish a standard specification provided by the present invention;
[0058] Figure 2 A flowchart of establishing a public opinion database in the method for processing public opinion data from multiple sources with a standard provided by the present invention;
[0059] Figure 3A flowchart of establishing a public opinion event data set in the method for processing public opinion data from multiple sources with a standard specification provided by the present invention;
[0060] Figure 4 A flowchart of a method for monitoring public opinion data in a method for processing public opinion data from multiple sources to establish a standard specification provided by the present invention;
[0061] Figure 5 A flowchart of a method for obtaining the emotional tendency of public opinion events in a multi-source public opinion data processing method for establishing a standard specification provided by the present invention. DETAILED DESCRIPTION
[0062] The following description is merely exemplary in nature and is not intended to limit the present invention, its application and use. It should be understood that in all of the accompanying drawings, the same or similar reference numerals indicate the same or similar parts and features. The various drawings only schematically represent the concepts and principles of the embodiments of the present invention, and do not necessarily show the specific dimensions and proportions of the various embodiments of the present invention. Specific parts in specific drawings may be exaggerated to illustrate the relevant details or structures of the embodiments of the present invention.
[0063] Reference Figure 1-Figure 5 :This embodiment provides a technical solution: Establish a standard multi-source public opinion data processing method, the specific steps are as follows:
[0064] Public opinion data acquisition: build a public opinion data acquisition channel, preset the data collection frequency, and collect public opinion data from the public opinion data acquisition channel at the preset frequency;
[0065] Establish a public opinion database: Analyze the acquired public opinion data and build a public opinion database;
[0066] Public opinion data monitoring: Analyze public opinion data and monitor specific public opinions;
[0067] Generate public opinion analysis report: form a standardized analysis report on public opinion of a specific event.
[0068] This embodiment provides a method for processing public opinion data from multiple sources with a standard specification. Specific channels for obtaining public opinion data include: internal data synchronization within a unit group, internal data sharing within a cooperative enterprise, and third-party public data capture. Third-party public data includes but is not limited to: social media data (such as Weibo, WeChat, Twitter), news website data, forum data, blog data, and comment platform data. The data collection tool uses one of the following: crawler technology, API interface, and data crawler tool. It should be noted that crawler technology is an automated network data collection method, which is mainly used in the following aspects:
[0069] Web crawling and data extraction: Automatically crawl required information from web pages by writing crawler programs, such as news, product information, social data, etc.
[0070] Website monitoring and surveillance: Use crawlers regularly to check the website's operating status, updates, traffic, etc., and identify problems in a timely manner;
[0071] Search engine optimization: Use crawlers to analyze the structure and content of a website to help it be better indexed and ranked by search engines;
[0072] Price comparison and e-commerce monitoring: crawling the price, inventory and other information of goods on different e-commerce platforms to provide users with price comparison and product monitoring services;
[0073] Public opinion monitoring and analysis: Analyze the public’s attitude and emotions towards a certain thing by crawling information from social media, news and other channels;
[0074] Crawler technology needs to combine programming languages, network protocols, data processing and other knowledge to design an efficient, legal and reliable data collection system; the rational use of crawlers can help obtain a large amount of valuable data resources;
[0075] An API (Application Programming Interface) is a set of protocols and tools that define how software components interact with each other. It allows communication and data exchange between different software systems, applications, or services.
[0076] APIs can be divided into several types:
[0077] Web API: An API accessed over the web using the HTTP protocol, typically returning data in JSON or XML format; common examples include RESTful API and SOAP API;
[0078] Library or framework API: A set of functions and tools provided in a specific programming language for developers to call when writing code; for example, Java's standard library API or Python's standard library;
[0079] Operating system API: An interface provided by the operating system that enables applications to interact with the underlying operating system, such as the Windows API or the POSIX standard;
[0080] The main functions of the API include:
[0081] Simplify development: Reduce the workload of developers by providing predefined functions;
[0082] System integration: allows integration between different systems or services to achieve data sharing and functional interoperability;
[0083] Modularity: Make software development more modular, easy to maintain and upgrade;
[0084] In short, API interfaces are an indispensable part of modern software development, which facilitates collaboration between different platforms and services;
[0085] Data scraping tools are tools used to automatically obtain data from websites or other online platforms. They can extract information from web pages and organize it into structured data. Common uses include data analysis, market research, competitive analysis, etc.
[0086] Common data scraping tools include:
[0087] Beautiful Soup: a Python library that is mainly used for parsing and extracting data from web pages; Scrapy: a powerful Python framework that supports large-scale web crawling; Selenium: although it is mainly used for automated testing, it can also be used to crawl dynamic web content; Octoparse: a code-free web crawler suitable for people who are not familiar with programming.
[0088] This embodiment provides a method for processing public opinion data from multiple sources with a standard specification. The specific method for establishing a public opinion database is as follows:
[0089] Public opinion data includes video data, text data, picture data, public comment data, and public likes data of public opinion events;
[0090] Identify the video data, text data and picture data in the public opinion data, and determine whether the collected video data, text data and picture data contain the same public opinion event. If it is determined that the collected video data, text data and picture data contain the same public opinion event, match, integrate and clean the video data, text data and picture data of the same public opinion event to form a public opinion data chain;
[0091] Determine whether the current public opinion data chain is associated with the historical public opinion data chain. If it is determined that the current public opinion data chain is associated with the historical public opinion data chain, merge the current public opinion data chain with the historical public opinion data chain, clean the duplicate parts, and establish a public opinion event data set;
[0092] Based on the initial time of the public opinion event, the public opinion event data set is arranged and integrated in chronological order to form a public opinion database.
[0093] This embodiment provides a method for processing multi-source public opinion data to establish standards and specifications. The specific method for determining whether there are the same public opinion events in the collected video data, text data, and picture data is as follows:
[0094] Extract the keywords in each video data, text data, and picture data. The keyword extraction adopts the following methods: term frequency analysis, TF-IDF, and topic model. It should be noted that term frequency analysis is a method of text analysis used to calculate and analyze the occurrence frequencies of various words in a text. This analysis can help understand the main themes of the text, identify keywords, and discover patterns or trends. Specifically, the steps of term frequency analysis usually include:
[0095] 1. Text cleaning: Remove unnecessary characters, punctuation marks, and stop words (such as words with no substantial meaning in the analysis like "de" and "shi").
[0096] 2. Word segmentation: Decompose the text into individual words or phrases.
[0097] 3. Frequency statistics: Calculate the number of times each word appears in the text.
[0098] 4. Result analysis: Analyze the key points or trends of the text content based on the high and low of the word frequencies.
[0099] Term frequency analysis is widely used in fields such as information retrieval, data mining, and sentiment analysis.
[0100] TF-IDF (Term Frequency-Inverse Document Frequency) is a commonly used information retrieval and text mining technique for evaluating the importance of a word or phrase in a document set (or corpus). The main uses of TF-IDF are:
[0101] 1. Determine keywords: Help extract and identify important keywords in the text.
[0102] 2. Calculate text similarity: Used to calculate the similarity between documents.
[0103] 3. Information retrieval: Help improve the relevance of search results in search engines.
[0104] Generally speaking, TF-IDF is a simple and effective text analysis method and is widely used in the fields of natural language processing and information retrieval.
[0105] Topic modeling is a natural language processing technique used to automatically extract and identify potential topics or themes from large amounts of text; the most common topic model is Latent Dirichlet Allocation (LDA), which can help discover major topics from text and assign documents to these topics;
[0106] The core idea of topic modeling is that each document is generated by several topics, and each topic is generated by a set of words. By analyzing the words and their distribution in a large number of documents, topic models can find out the potential topics in the documents and their related words.
[0107] This technology is widely used in fields such as text analysis, information retrieval and recommendation systems.
[0108] Based on the extracted keywords, create labels for each video data, text data and image data. The labels include: the time when the public opinion event occurred, the location of the public opinion event, and the description of the public opinion event;
[0109] The labels of each video data, text data and image data are compared. When the compared labels are consistent, they are determined to be the same public opinion event.
[0110] This embodiment provides a method for processing public opinion data from multiple sources with a standard specification. Specifically, the steps for matching, integrating and cleaning the video data, text data and picture data of the same public opinion event to form a public opinion data chain are as follows:
[0111] Combine the video data, text data and picture data, public comment data and public likes data of the same public opinion event, and arrange them in a preset order, such as: arrange in the order of video data-picture data-text data-public comment data-public likes data or text data-picture data-video data-public comment data-public likes data;
[0112] The repeated and invalid parts in the video data, text data, image data and public comment data are eliminated to obtain a uniquely constructed public opinion data chain.
[0113] This embodiment provides a method for processing public opinion data from multiple sources with a standard specification. The specific method for establishing a public opinion event data set is as follows:
[0114] Compare the label of the current public opinion data chain with the label of the historical public opinion data chain. When the compared labels are consistent, it is determined that the current public opinion data chain is associated with the historical public opinion data chain.
[0115] The current public opinion data chain is merged with the historical public opinion data chain, the duplicate parts in the merged public opinion data chain are removed, and they are rearranged in a preset order to establish a public opinion event data set.
[0116] This embodiment provides a method for processing public opinion data from multiple sources with a standard specification. The specific method for monitoring public opinion data is as follows:
[0117] Parsing the public opinion event dataset in the public opinion database:
[0118] Conduct sentiment analysis on the public opinion event data set to obtain the sentiment tendency of the public opinion events;
[0119] Monitoring specific public opinions:
[0120] Calculate the heat value of the public opinion event data set;
[0121] Preset the heat value monitoring threshold and perform tagging monitoring on the public opinion event data set that exceeds the heat monitoring threshold;
[0122] Establish an early warning mechanism based on the public opinion event dataset with spiked monitoring.
[0123] This embodiment provides a method for processing public opinion data from multiple sources with a standard specification. The specific method for obtaining the emotional tendency of a public opinion event through emotional analysis is as follows:
[0124] Extract public comment data from the public opinion event dataset;
[0125] Analyze the sentiment of each public comment through natural language processing technology. The sentiment tendencies include positive sentiment tendency, negative sentiment tendency, and neutral sentiment tendency.
[0126] Count the sentiment trends of all public comments and obtain the number of comments with positive sentiment trends, negative sentiment trends, and neutral sentiment trends respectively;
[0127] Based on the sentiment trends of all public comments, the sentiment trends of public opinion events are judged. The specific judgment method is as follows:
[0128] By comparing the emotional tendencies of all public comments, when the number of comments with positive emotional tendencies is the largest, the public opinion event is judged to have a positive emotional tendency; similarly, when the number of comments with negative emotional tendencies is the largest, the public opinion event is judged to have a negative emotional tendency; when the number of comments with neutral emotional tendencies is the largest, the public opinion event is judged to have a neutral emotional tendency.
[0129] This embodiment provides a method for processing public opinion data from multiple sources with a standard specification. The specific method for calculating the heat value of a public opinion event data set is as follows:
[0130] ;
[0131] in, is the heat value of the public opinion event dataset, Extracted from the public opinion event dataset Number of public comments in data collection channels, Extracted from the public opinion event dataset Number of public likes in the data collection channels, , The weight of the impact of public comments and public likes on the popularity value, , are greater than 0, , A technician in this field collects multiple sets of comprehensive parameters, sets corresponding weight coefficients for each set of comprehensive parameters, substitutes the set weight coefficients and the collected comprehensive parameters into the formula, and screens and averages the calculated weight coefficients to obtain , The mean of , optionally, =0.68, =0.32.
[0132] The method of establishing an early warning mechanism based on the public opinion event dataset with spiked monitoring is as follows:
[0133] The heat value change collection time period is preset, and the heat value of the public opinion event data set monitored by the spike in each time period is collected, which are recorded as , ,…… ,in, For the The heat value of the public opinion event data set monitored at a certain time period;
[0134] Count the changes in heat value within each time period ,but: ,in, For the The heat value of the public opinion event data set monitored by the spiked monitoring in the time period, For the The heat value of the public opinion event data set monitored at a certain time period;
[0135] Mark the preset heat value change threshold as ,
[0136] like < , then no warning instruction is generated;
[0137] like ≥ , then a warning instruction is generated, and based on the warning instruction, a warning information is issued to the platform. Among them, the preset heat value change threshold is the basis for dividing the generation of warning instructions. The setting of the heat value change threshold is through collecting multiple groups of different public opinion event data information in historical situations, and combined with the influence of public opinion events. Set it, so that the heat value change can reflect the changing trend of public opinion events.
[0138] This embodiment provides a method for processing public opinion data from multiple sources with a standard specification. The specific method for generating a public opinion analysis report is as follows:
[0139] Upload the public opinion standardized analysis report template;
[0140] Based on the specific public opinion being monitored, extract the data information in the public opinion event data set, write it into the corresponding public opinion standardized analysis report template, and form a standardized analysis report.
[0141] In summary, the method for processing public opinion data from multiple sources with a standard specification provided by the present invention can obtain public opinion data from multiple channels, and analyze and integrate the public opinion data obtained from multiple channels, so as to understand the public's emotional tendency towards public opinion events, so as to facilitate subsequent manual intervention and guidance when the negative emotional tendency generated by public opinion events causes negative situations among the public, so as to prevent the occurrence of sudden events;
[0142] This method can be used to analyze public opinion data from multiple channels, integrate public opinion data from multiple channels, calculate the heat value of public opinion events, monitor public opinion events with higher heat values (higher heat values indicate greater influence of public opinion events), and facilitate real-time understanding of the trends of key public opinion events.
[0143] The exemplary implementation scheme of the present invention is described in detail above with reference to the preferred embodiments. However, those skilled in the art will appreciate that, without departing from the concept of the present invention, various modifications and variations may be made to the above-mentioned specific embodiments, and various technical features and structures proposed in the present invention may be combined in various ways without exceeding the protection scope of the present invention, which is determined by the appended claims.
Claims
1. Establish a standardized method for processing public opinion data from multiple sources, characterized by: The specific steps are as follows: Public opinion data acquisition: build a public opinion data acquisition channel, preset the data collection frequency, and collect public opinion data from the public opinion data acquisition channel at the preset frequency; Establish a public opinion database: Analyze the acquired public opinion data and build a public opinion database; Public opinion data monitoring: Analyze public opinion data and monitor specific public opinions; Generate public opinion analysis report: form a standardized analysis report on public opinion of a specific event.
2. The method for processing public opinion data from multiple sources according to establishing a standard specification according to claim 1 is characterized in that: The public opinion data acquisition channels include: internal data synchronization of unit groups, internal data sharing of cooperative enterprises, and capture of third-party public data.
3. The method for processing public opinion data from multiple sources according to establishing a standard specification according to claim 1 is characterized in that: The specific methods for establishing a public opinion database are as follows: Public opinion data includes video data, text data, picture data, public comment data, and public likes data of public opinion events; Identify the video data, text data and picture data in the public opinion data, and determine whether the collected video data, text data and picture data contain the same public opinion event. If it is determined that the collected video data, text data and picture data contain the same public opinion event, match, integrate and clean the video data, text data and picture data of the same public opinion event to form a public opinion data chain; Determine whether the current public opinion data chain is associated with the historical public opinion data chain. If it is determined that the current public opinion data chain is associated with the historical public opinion data chain, merge the current public opinion data chain with the historical public opinion data chain, clean the duplicate parts, and establish a public opinion event data set; Based on the initial time of the public opinion event, the public opinion event data set is arranged and integrated in chronological order to form a public opinion database.
4. The method for processing public opinion data from multiple sources according to claim 3, characterized in that: The method for determining whether the collected video data, text data, and image data contain the same public opinion event is as follows: Extract keywords from each video data, text data and picture data, wherein the keyword extraction adopts the following methods: word frequency analysis, TF-IDF, topic model; Based on the extracted keywords, create labels for each video data, text data and image data, the labels including: the time when the public opinion event occurred, the location where the public opinion event occurred, and the description of the public opinion event; The labels of each video data, text data and image data are compared. When the compared labels are consistent, they are determined to be the same public opinion event.
5. The method for processing public opinion data from multiple sources according to claim 4, characterized in that: The specific steps for matching, integrating and cleaning the video data, text data and image data of the same public opinion event to form a public opinion data chain are as follows: Combine the video data, text data, picture data, public comment data, and public likes data of the same public opinion event, and arrange them in a preset order; The repeated and invalid parts in the video data, text data, image data and public comment data are eliminated to obtain a uniquely constructed public opinion data chain.
6. The method for processing public opinion data from multiple sources according to establishing a standard specification according to claim 3 is characterized in that: The specific method of establishing a public opinion event dataset is as follows: Compare the label of the current public opinion data chain with the label of the historical public opinion data chain. When the compared labels are consistent, it is determined that the current public opinion data chain is associated with the historical public opinion data chain. The current public opinion data chain is merged with the historical public opinion data chain, the duplicate parts in the merged public opinion data chain are removed, and they are rearranged in a preset order to establish a public opinion event data set.
7. The method for processing public opinion data from multiple sources according to establishing a standard specification according to claim 1 is characterized in that: The method for monitoring public opinion data is as follows: Parsing the public opinion event dataset in the public opinion database: Conduct sentiment analysis on the public opinion event data set to obtain the sentiment tendency of the public opinion events; Monitoring specific public opinions: Calculate the heat value of the public opinion event dataset; Preset the heat value monitoring threshold and perform tagging monitoring on the public opinion event data set that exceeds the heat monitoring threshold; Establish an early warning mechanism based on the public opinion event dataset with spiked monitoring.
8. The method for processing public opinion data from multiple sources according to claim 7, characterized in that: The method of obtaining the emotional tendency of public opinion events through sentiment analysis is as follows: Extract public comment data from the public opinion event dataset; Analyze the sentiment tendency of each public comment through natural language processing technology, wherein the sentiment tendency includes: positive sentiment tendency, negative sentiment tendency, and neutral sentiment tendency; Count the sentiment trends of all public comments and obtain the number of comments with positive sentiment trends, negative sentiment trends, and neutral sentiment trends respectively; Based on the sentiment tendencies of all statistical public comments, judge the sentiment tendencies of public opinion events.
9. The method for processing public opinion data from multiple sources according to claim 7, characterized in that: The method for calculating the heat value of the public opinion event dataset is as follows: ; in, is the heat value of the public opinion event dataset, Extracted from the public opinion event dataset Number of public comments in data collection channels, Extracted from the public opinion event dataset Number of public likes in the data collection channels, , The weight of the impact of public comments and public likes on the popularity value; The method of establishing an early warning mechanism based on the public opinion event dataset with spiked monitoring is as follows: The heat value change collection time period is preset, and the heat value of the public opinion event data set monitored by the spike in each time period is collected, which are recorded as , ,…… ,in, For the The heat value of the public opinion event data set monitored at a certain time period; Count the changes in heat value within each time period ,but: ,in, For the The heat value of the public opinion event data set monitored by the spiked monitoring in the time period, For the The heat value of the public opinion event data set monitored at a certain time period; Mark the preset heat value change threshold as , like < , no warning instruction is generated; like ≥ , then an early warning instruction is generated, and based on the early warning instruction, an early warning message is sent to the platform.
10. The method for processing public opinion data from multiple sources according to claim 1, characterized in that: The method to generate a public opinion analysis report is as follows: Upload the public opinion standardized analysis report template; Based on the specific public opinion being monitored, extract the data information in the public opinion event data set, write it into the corresponding public opinion standardized analysis report template, and form a standardized analysis report.
Citation Information
Patent Citations
Public opinion monitoring method and apparatus
CN106126558A
Video-based event evolution prediction method and system
CN113486754A
Internet big data analysis method and system
CN114638230A
Network public opinion monitoring method and system and storage medium
CN117609481A
