Network public opinion prediction method and system based on deep learning
Through the deep learning-based online public opinion prediction method, combined with emotional knowledge graph and graph embedding technology, the problem that the existing technology cannot handle complex emotional expression and fine-grained emotional changes is solved, high-precision and real-time public opinion monitoring are achieved, and decision-making is improved timeliness and accuracy.
Patent Information
- Application Number
- CN202410993602.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-12-03
AI Technical Summary
The prior art cannot effectively deal with complex emotional expressions and fine-grained emotional changes, especially in large-scale real-time data, which makes it difficult to capture rapidly changing public opinion trends, limiting the timeliness and accuracy of decision-making.
Using a deep learning-based network public opinion prediction method, collect text data from social media platforms, preprocess and build emotional knowledge graphs, use graph convolution networks to embed graphs, combine traditional text features and graph embed features, and use LSTM or Transformer The model is trained in deep learning to achieve emotional prediction and public opinion trend analysis.
Improves the accuracy and real-time monitoring capabilities of sentiment analysis, enables deep understanding of complex semantic relationships and emotional expressions in text, provides higher accuracy and more comprehensive sentiment and semantic analysis than traditional methods, ensuring that the system is dealing with large-scale High performance and low latency when data.
Smart Images

Figure CN119066272A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of network public opinion prediction, and specifically to a network public opinion prediction method and system based on deep learning. Background Art
[0002] In today's rapidly developing information society, social media platforms have become the main channel for the public to express their emotions and opinions. With the surge in data volume, how to effectively extract and analyze public opinion from huge social media data has become the focus of attention of governments, enterprises and other organizations. This demand for real-time public opinion monitoring and analysis has promoted the development of public opinion prediction systems based on deep learning technology.
[0003] Existing techniques usually rely on traditional text analysis methods, such as keyword search and basic sentiment analysis, which can provide a certain degree of emotional overview. However, most of them rely on simple algorithms, such as word frequency statistics or basic natural language processing techniques, to identify positive or negative emotions in text.
[0004] Although these existing technologies perform well in some scenarios, they are usually unable to handle complex emotional expressions and fine-grained emotional changes, especially in terms of the diversity of implicit semantics and emotions in text data. In addition, traditional methods often have difficulty capturing rapidly changing public opinion trends when processing large-scale real-time data, which limits the timeliness and accuracy of decision-making. Therefore, an advanced public opinion prediction system that can effectively solve these problems is needed to improve the accuracy of sentiment analysis and real-time monitoring capabilities. Summary of the invention
[0005] In view of the deficiencies in the prior art, the present invention provides a method and system for predicting online public opinion based on deep learning, which solves the problem of being unable to effectively handle complex emotional expressions and fine-grained emotional changes.
[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a network public opinion prediction method based on deep learning, comprising:
[0007] Step 1: Collect text data from multiple social media platforms;
[0008] Step 2: Preprocessing the text data, wherein the preprocessing includes text cleaning and standardization.
[0009] Step 3: constructing a sentiment knowledge graph based on the preprocessed text, wherein the graph includes entity recognition, relationship extraction and sentiment attribute analysis;
[0010] Step 4: Using a graph convolutional network to embed the sentiment knowledge graph to obtain a low-dimensional vector representation of the node;
[0011] Step 5: Combine traditional text features and graph embedding features to perform deep learning training using a long short-term memory network or a Transformer model;
[0012] Step 6: Based on the training results, perform sentiment prediction and public opinion trend analysis on new social media data.
[0013] Preferably, the data collection step further comprises using APIs to automatically collect posts containing specific keywords from social media platforms, Twitter, Facebook and Weibo.
[0014] Preferably, the text preprocessing step further includes using a natural language processing tool, spaCy or NLTK, to perform automated word segmentation, stop word removal and stem extraction.
[0015] Preferably, the step of constructing the sentiment knowledge graph further includes using a specific natural language processing model to perform a fine-grained analysis of the sentiment tendency and intensity in the text, and integrating this information into the graph.
[0016] Preferably, the formula in the graph embedding step is:
[0017]
[0018] It further includes selecting the activation function σ as ReLU or tanh to improve the nonlinear expression ability of graph embedding.
[0019] Preferably, the LSTM or Transformer model training step further includes applying an incremental learning strategy on data collected at multiple time points to adjust model parameters in real time in response to changes in social media trends.
[0020] Preferably, the emotion prediction step further comprises calculating the emotion score S using the following formula: i , considering the sentiment tendency and context relevance of a single message:
[0021] S i =β˙context score+(1-β)˙emotion score
[0022] Among them, β is a parameter that adjusts the influence of context relevance on the total score.
[0023] A network public opinion prediction system based on deep learning, comprising:
[0024] a data collection module configured to collect text data containing specific keywords from multiple social media platforms via APIs;
[0025] A data processing module, configured to receive data from the data collection module and perform preprocessing, including text cleaning and standardization;
[0026] A sentiment knowledge graph construction module, configured to receive the preprocessed text from the data processing module and construct a sentiment knowledge graph based on the text, including entity recognition, relationship extraction and sentiment attribute analysis;
[0027] a graph embedding module configured to process the sentiment knowledge graph from the sentiment knowledge graph building module using a graph convolutional network to obtain a low-dimensional vector representation of a node;
[0028] A deep learning module, configured to receive the low-dimensional vector representation from the graph embedding module and the traditional text features from the data processing module, trained using an LSTM or Transformer model;
[0029] A predictive analysis module, configured to perform sentiment prediction and public opinion trend analysis on new social media data based on the training results from the deep learning module, and generate a predictive report
[0030] Preferably, the data collection module is configured to automatically update the data set according to keywords and time intervals set by the user, and the prediction analysis module is configured to output a visual report containing sentiment tendency, sentiment intensity and predicted trend.
[0031] Preferably, each module communicates via APIs to ensure real-time processing and analysis of data.
[0032] The present invention provides a method and system for predicting online public opinion based on deep learning. It has the following beneficial effects:
[0033] 1. By combining traditional text analysis with advanced graph embedding technology, the system can deeply understand the complex semantic relationships and emotional expressions in the text. The emotional knowledge graph construction module and graph embedding module enable the system to accurately identify and analyze subtle emotional changes in the text, providing higher accuracy than traditional sentiment analysis methods.
[0034] 2. The present invention has designed an efficient data collection and real-time processing process, which can continuously monitor the public opinion dynamics on social media platforms. Through real-time data analysis and trend prediction, organizations can quickly respond to changes in public sentiment and adjust public relations strategies or respond to potential crises in a timely manner.
[0035] 3. The present invention uses advanced graph convolutional networks (GCN) and deep learning techniques, such as LSTM or Transformer, to effectively process large-scale and high-dimensional social media data. The application of these technologies ensures high performance and low latency of the system when processing large amounts of data.
[0036] 4. The present invention can also analyze the associated entities and the relationships between them in the text data through the sentiment knowledge graph, thereby providing a more comprehensive sentiment and semantic analysis. This multi-dimensional analysis method makes the public opinion report more comprehensive and in-depth. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 is a flow chart of the method of the present invention;
[0038] Figure 2 It is a system schematic diagram of the present invention. DETAILED DESCRIPTION
[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0040] Embodiment 1:
[0041] Please see attached Figure 1 The embodiment of the present invention provides a method for predicting network public opinion based on deep learning, comprising the following steps:
[0042] In this embodiment, the data collection step involves automatically obtaining text data containing specific keywords from multiple social media platforms through APIs. The specific operations are as follows:
[0043] Use open APIs from Twitter, Facebook, and Weibo and register your app to gain access.
[0044] Set up a list of keywords that are related to the events or topics you want to monitor.
[0045] Use scheduled tasks (such as Cron jobs) to automatically trigger data collection scripts to ensure that the latest data is collected at a certain frequency (for example, once an hour).
[0046] The collected data should include the text content of the post, publishing time, author information, etc., and stored in the database for subsequent processing.
[0047] Data preprocessing
[0048] In this embodiment, the data preprocessing step includes text cleaning and standardization, and the operations are as follows:
[0049] Text cleaning: Use natural language processing tools (such as Python's regular expression library) to remove URLs, HTML tags, special symbols, and non-text content from the text.
[0050] Text normalization: convert all text to lowercase, tokenize using a natural language processing tool such as NLTK or spaCy, remove stop words, and perform stemming.
[0051] The preprocessed text data are stored in a database and formatted into a structure suitable for subsequent analysis.
[0052] Emotional knowledge graph construction
[0053] In this embodiment, constructing a sentiment knowledge graph includes entity recognition, relationship extraction, and sentiment attribute analysis. The specific operations are as follows:
[0054] Entity Recognition: Use natural language processing tools such as BERT or spaCy to identify key entities in text, such as names of people, places, and organizations.
[0055] Relation extraction: Determine the relationship type between entities (such as causal and contrastive relationships) through dependency syntactic analysis.
[0056] Sentiment attribute analysis: Apply sentiment analysis models (such as TextBlob or self-trained models) to evaluate the sentiment tendency (positive, negative, neutral) and strength of entities and relations.
[0057] Use a graph database (such as Neo4j) to store the sentiment knowledge graph, where nodes represent entities and relationships, edges represent connections between entities, and the attributes of nodes and edges contain sentiment information.
[0058] Graph Embedding
[0059] In this embodiment, the graph embedding step uses a graph convolutional network (GCN) to convert the sentiment knowledge graph into a low-dimensional vector representation. The specific operations are as follows:
[0060] Use a GCN library (such as PyTorch Geometry) to process the graph data and extract feature vectors.
[0061] Configure the number of layers, number of hidden units, and activation function (usually ReLU or tanh) of the GCN to optimize performance and accuracy.
[0062] The graph embedding results are stored as vectors for use by deep learning models.
[0063] In the graph embedding step, a graph convolutional network (GCN) is used to process the sentiment knowledge graph and transform entities and relationships into low-dimensional vector representations. The key formula is as follows:
[0064]
[0065] h v (l) Represents the feature vector of node v at layer l.
[0066] σ is a non-linear activation function such as ReLU or tanh.
[0067] N(v) is the set of neighbor nodes of node v.
[0068] c vu is a normalization constant, and a common choice is the number of neighbors of node v to ensure the stability of training.
[0069] W (l) and B (l) are the weight matrix and bias vector of the lth layer respectively.
[0070] This formula enables the model to learn high-order feature representations of nodes by applying convolution operations on graph-structured data. These features are rich in neighborhood information and are crucial for subsequent sentiment analysis.
[0071] Deep Learning Training
[0072] In this embodiment, the LSTM or Transformer model is used to combine traditional text features and graph embedding features for training. The specific operations are as follows:
[0073] Combine traditional text features such as TF-IDF with the vector features obtained from the graph embedding step.
[0074] Configure and train LSTM or Transformer models using a deep learning framework such as TensorFlow or PyTorch.
[0075] Train, validate and test the model, and use the cross-validation method to optimize the model parameters.
[0076] Sentiment prediction and public opinion trend analysis
[0077] In this embodiment, the steps of sentiment prediction and public opinion trend analysis are as follows:
[0078] Apply trained models to perform real-time sentiment analysis on newly collected social media data.
[0079] Use time series analysis methods (such as the ARIMA model) to predict public opinion trends based on historical sentiment data.
[0080] The results can be dynamically presented through visualization tools (such as Tableau or Python's matplotlib library) to assist decision makers in interpretation and decision-making.
[0081] In the sentiment prediction step, the trained model is combined with historical public opinion data to predict future trends. The key sentiment prediction model formula is as follows:
[0082]
[0083] S t is the predicted public opinion score at time t.
[0084] S t-1 is the public opinion score of the previous time step.
[0085] α is a smoothing parameter used to adjust the weight of historical public opinion scores in new predictions.
[0086] N t is the number of social media posts considered at time t.
[0087] w i is the weight of the ith post, which can be determined based on the author’s influence or the reach of the post.
[0088] s i is the sentiment score of the ith post, obtained through the sentiment analysis model.
[0089] This formula combines historical trends and weighted sentiment scores of the latest data to effectively predict changing trends in online public opinion and provide a scientific basis for decision-making.
[0090] Embodiment 2:
[0091] Please see attached Figure 2 The embodiment of the present invention provides a network public opinion prediction system based on deep learning, comprising:
[0092] Data collection module
[0093] In this embodiment, the data collection module automatically grabs data from social media platforms through customized scripts. The specific implementation of this module is as follows:
[0094] API interface calls: Use Twitter’s Tweepy library, Facebook’s Graph API, and Weibo’s open platform API to set appropriate access frequency and data crawling scope according to the limitations of the application programming interface (API).
[0095] Dynamic keyword configuration: The user interface allows users to enter or modify keywords, which will be used to filter content on social media. Keywords can be words related to current hot events or specific nouns or phrases.
[0096] Data preprocessing: Before data storage, the raw data is preprocessed by scripts, including the unification of timestamps, anonymization of author information, and filtering of non-text content.
[0097] Data storage: The processed data is stored in a relational database or NoSQL database, such as MySQL or MongoDB, to ensure data integrity and query efficiency.
[0098] Data processing module
[0099] The data processing module is responsible for extracting useful information from the raw data and preparing structured data for subsequent analysis. The specific implementation includes:
[0100] Text cleaning: Apply regular expressions to process text, remove useless symbols, numbers, and links. Use NLTK or spaCy libraries to remove stop words and perform part-of-speech restoration.
[0101] Feature extraction: Extract features from the cleaned text and use the TF-IDF method to convert the text data into vector form so that the machine learning model can process it.
[0102] Data standardization: Standardize all text features to ensure balanced weights between different features and improve the accuracy of model training.
[0103] Emotional knowledge graph building module
[0104] The emotional knowledge graph construction module is the core part of the system, which is responsible for identifying the emotional and semantic relationships in the text and building the graph. The specific implementation methods include:
[0105] Entity Recognition: Use the pre-trained BERT model to identify key entities such as people, places, and events from text.
[0106] Relation extraction: Use dependency parsing techniques to identify relationships between entities, such as "support", "oppose", "mention", etc.
[0107] Sentiment Analysis: Apply a sentiment analysis model, such as VADER or a self-trained sentiment classification model, to each entity and relationship to determine its sentiment polarity and intensity.
[0108] Graph construction: Store the identified entities and relationships in the form of a graph, using a graph database such as Neo4j, where nodes represent entities or relationships and edges represent connections between entities.
[0109] Graph Embedding Module
[0110] The graph embedding module uses a graph convolutional network (GCN) to convert graph data into feature vectors that can be used by deep learning models. Specific implementations include:
[0111] Graph data preprocessing: Convert the data in the graph database into the format required for GCN processing, including the adjacency matrix and node feature matrix.
[0112] GCN training: Configure the parameters of each layer of the GCN model, choose a suitable activation function (such as ReLU), and train the model to learn effective node embeddings.
[0113] Feature vector extraction: Extract the feature vector of the node from the trained GCN model as the input of the subsequent deep learning module.
[0114] Deep Learning Module
[0115] The deep learning module combines traditional text features and graph embedding features, and uses deep learning technology to perform sentiment prediction and trend analysis. Specific implementation methods include:
[0116] Model selection: Select LSTM or Transformer as the core model based on the data characteristics to process sequence data and capture long-term dependencies.
[0117] Feature fusion: The TF-IDF vector extracted from the data processing module is combined with the node vector of the graph embedding module to form a comprehensive feature.
[0118] Model training and validation: Train and validate the model on historical data sets, use cross-validation methods to optimize model parameters, and improve the generalization ability of the model.
[0119] Predictive analysis module
[0120] The prediction and analysis module is responsible for applying the trained model to new data to achieve real-time sentiment prediction and public opinion trend analysis. Specific implementation methods include:
[0121] Real-time data processing: Apply the same preprocessing and feature extraction pipeline to data collected in real time.
[0122] Sentiment prediction: Use the trained deep learning model to perform sentiment analysis on new data and predict its sentiment tendency.
[0123] Trend analysis: Combine historical sentiment data with real-time forecast results, use statistical methods or machine learning models to analyze public opinion trends and identify potential change patterns.
[0124] Results presentation: Develop an interactive user interface to display sentiment prediction and trend analysis results in real time, and support graphical presentation of data, such as trend graphs and heat maps, to assist users in making decisions.
[0125] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A network public opinion prediction method based on deep learning, characterized in that: include: Step 1: Collect text data from multiple social media platforms; Step 2: Preprocessing the text data, wherein the preprocessing includes text cleaning and standardization. Step 3: constructing a sentiment knowledge graph based on the preprocessed text, wherein the graph includes entity recognition, relationship extraction and sentiment attribute analysis; Step 4: Using a graph convolutional network to embed the sentiment knowledge graph to obtain a low-dimensional vector representation of the node; Step 5: Combine traditional text features and graph embedding features to perform deep learning training using a long short-term memory network or a Transformer model; Step 6: Based on the training results, perform sentiment prediction and public opinion trend analysis on new social media data.
2. According to the method for predicting network public opinion based on deep learning in claim 1, it is characterized in that: The data collection step further includes using APIs to automatically collect posts containing specific keywords from social media platforms, Twitter, Facebook, and Weibo.
3. According to the method for predicting network public opinion based on deep learning in claim 1, it is characterized in that: The text preprocessing step further includes using natural language processing tools, such as spaCy or NLTK, to perform automated word segmentation, stop word removal, and stem extraction.
4. According to the method for predicting network public opinion based on deep learning in claim 1, it is characterized in that: The step of constructing the sentiment knowledge graph further includes using a specific natural language processing model to perform a fine-grained analysis of the sentiment tendency and intensity in the text and integrate this information into the graph.
5. According to the method for predicting network public opinion based on deep learning in claim 1, it is characterized in that: The formula in the graph embedding step is: It further includes selecting the activation function σ as ReLU or tanh to improve the nonlinear expression ability of graph embedding.
6. According to the method of predicting network public opinion based on deep learning in claim 1, it is characterized in that: The LSTM or Transformer model training step further includes applying an incremental learning strategy on data collected at multiple time points to adjust model parameters in real time in response to changes in social media trends.
7. According to the method of predicting network public opinion based on deep learning in claim 1, it is characterized in that: The emotion prediction step further includes calculating the emotion score S using the following formula i , considering the sentiment tendency and context relevance of a single message: S i =β˙context score+(1-β)˙emotion score Among them, β is a parameter that adjusts the influence of context relevance on the total score.
8. A network public opinion prediction system based on deep learning, according to a network public opinion prediction method based on deep learning according to any one of claims 1-9, characterized in that: include: a data collection module configured to collect text data containing specific keywords from multiple social media platforms via APIs; A data processing module, configured to receive data from the data collection module and perform preprocessing, including text cleaning and standardization; A sentiment knowledge graph construction module, configured to receive the preprocessed text from the data processing module and construct a sentiment knowledge graph based on the text, including entity recognition, relationship extraction and sentiment attribute analysis; a graph embedding module configured to process the sentiment knowledge graph from the sentiment knowledge graph building module using a graph convolutional network to obtain a low-dimensional vector representation of a node; A deep learning module, configured to receive the low-dimensional vector representation from the graph embedding module and the traditional text features from the data processing module, trained using an LSTM or Transformer model; The predictive analysis module is configured to perform sentiment prediction and public opinion trend analysis on new social media data based on the training results from the deep learning module, and generate a predictive report.
9. The network public opinion prediction system based on deep learning according to claim 8, characterized in that: The data collection module is configured to automatically update the data set according to the keywords and time intervals set by the user, and the prediction analysis module is configured to output a visual report containing sentiment tendency, sentiment intensity and prediction trend.
10. The network public opinion prediction system based on deep learning according to claim 8, characterized in that: Each module communicates via APIs, ensuring real-time processing and analysis of data.
Citation Information
Cited By
Data acquisition and processing method, device, equipment and product
CN120353779A
Regionalization analysis-based scientific culture transmission feasibility evaluation system
CN120833005A
