Public opinion analysis method, system and equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and particularly provides a public opinion analysis method, system and device and a storage medium, and the method comprises the steps: collecting user evaluation data, and carrying out the preprocessing of the user evaluation data; utilizing a pre-training language model and a topic model to analyze the preprocessed public opinion topic of the user evaluation data; grouping the preprocessed user evaluation data according to public opinion themes to which the user evaluation data belong to obtain a plurality of data groups; and utilizing an emotion analysis model to respectively identify the emotion tendencies of the plurality of data groups to obtain the emotion tendencies corresponding to the plurality of public opinion topics, the emotion analysis model comprising a pre-training language model, a bidirectional long-short term memory network model and a convolutional neural model. According to the method, hierarchical analysis of the user evaluation data is realized, and data support is provided for subsequent decision making.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a public opinion analysis method, system, device and storage medium. Background Art

[0002] Public opinion analysis plays an extremely important role in today's society. Through public opinion analysis, targeted decision-making can be made. Public opinion information comes from a wide range of sources, covering multiple media platforms and social media, such as news, Weibo, forums, social networks, etc. Due to the huge amount of information and the diverse content involved, it may be difficult to collect and integrate public opinion information. The quality of public opinion information is unstable, including a large amount of user-generated content, with varying quality and authenticity. It is a challenge to filter out truly valuable and credible public opinion information from the massive amount of information.

[0003] Existing tourism public opinion monitoring systems mainly use traditional machine learning algorithms for natural language processing, including support vector machines (SVM), Naive Bayes, Random Forest, etc. These methods usually require manual feature design and extracting text features such as bag-of-words models and TF-IDF during the feature engineering process, and then use machine learning algorithms for sentiment classification.

[0004] In the era of big data, massive amounts of data with more diverse forms often pose challenges to these traditional methods. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a public opinion analysis method, system, device and storage medium to solve the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a public opinion analysis method, comprising: Collecting user evaluation data and preprocessing the user evaluation data; Use pre-trained language models and topic models to analyze public opinion topics of pre-processed user evaluation data; The preprocessed user evaluation data is grouped according to the public opinion topics to which they belong, to obtain multiple data groups; The sentiment analysis model is used to identify the sentiment tendencies of the multiple data groups respectively, and the sentiment tendencies corresponding to the multiple public opinion topics are obtained. The sentiment analysis model includes a pre-trained language model, a bidirectional long short-term memory network model and a convolutional neural model.

[0007] In an optional implementation, collecting user evaluation data and preprocessing the user evaluation data includes: Use crawler technology to crawl user evaluation data of target objects from multiple platforms, and save the crawled user evaluation data into a folder pre-built for the corresponding platform; Cleaning the user evaluation data to remove invalid fields and escape characters in the user evaluation data; Use word segmentation tools and custom dictionaries to segment sentences in the cleaned user evaluation data to obtain word sequences.

[0008] In an optional implementation, the pre-trained language model and the topic model are used to analyze the public opinion topics of the pre-processed user evaluation data, including: Extracting a word sequence of user evaluation data in a target folder, inputting the word sequence into a pre-trained language model, and obtaining a corresponding word vector set; Calculate the similarity between word vectors in the word vector set, mark the word vectors whose similarity exceeds the set similarity threshold as duplicate word vectors, and perform deduplication processing on the word vector set to obtain a streamlined word vector set; The simplified word vector set is input into the topic model to obtain the public opinion topic and the distribution of the public opinion topic in the text.

[0009] In an optional implementation, the pre-processed user evaluation data is grouped according to the public opinion topic to which it belongs, to obtain multiple data groups, including: According to the vocabulary distribution of each topic in the topic model, the topic is named and explained, and the corresponding topic label is generated for the topic based on the naming and explanation of the topic; By calculating the similarity between each user evaluation data and each topic, one or more topic labels are assigned to each user evaluation data; User evaluation data with the same topic label are divided into the same data group.

[0010] In an optional implementation, the sentiment analysis model includes: A pre-trained language model includes an input layer, an encoding layer and an output layer. The input layer is pre-processed by three-layer vectorization of word vector, text vector and position vector. The encoding layer is formed by accumulating 24 bidirectional Transformer encoders. The internal structure includes 1024 hidden layers and 16 attention mechanisms. The output layer outputs a word vector matrix. The pre-trained language model is fine-tuned using a masked language model and a context prediction task. A bidirectional long short-term memory network model, wherein the bidirectional long short-term memory network model uses a bidirectional encoding mode to capture feature vectors of the word vector matrix, and concatenates the captured feature vectors into output features; A convolutional neural model is provided, wherein the convolutional neural model performs local feature recognition on the output features, obtains the association between adjacent words, applies a maximum pooling layer to update and reduce key words, outputs the final key words to a fully connected layer, uses an activation function to perform nonlinear processing on the key words, and uses a softmax classifier to perform sentiment polarity classification on the key words after the nonlinear processing.

[0011] In an optional embodiment, the method further comprises: Use sentiment analysis models to perform sentiment analysis on all data groups, and calculate sentiment tendencies based on sentiment analysis results; Based on the correspondence between the data group to which the sentiment tendency belongs and the public opinion topic, a correspondence between the public opinion topic and the sentiment tendency is established.

[0012] In a second aspect, the present invention provides a public opinion analysis system, comprising: A data collection module, used to collect user evaluation data and pre-process the user evaluation data; The topic analysis module is used to analyze the public opinion topics of the pre-processed user evaluation data using the pre-trained language model and the topic model; A data grouping module is used to group the pre-processed user evaluation data according to the public opinion topics to which they belong, to obtain multiple data groups; The sentiment analysis module is used to use the sentiment analysis model to respectively identify the sentiment tendencies of the multiple data groups and obtain the sentiment tendencies corresponding to the multiple public opinion topics. The sentiment analysis model includes a pre-trained language model, a bidirectional long short-term memory network model and a convolutional neural model.

[0013] In a third aspect, a device is provided, including: A memory, used for storing a public opinion analysis program; A processor is used to implement the steps of the public opinion analysis method provided in the first aspect when executing the public opinion analysis program.

[0014] In a fourth aspect, a computer-readable storage medium is provided, on which a public opinion analysis program is stored. When the public opinion analysis program is executed by a processor, the steps of the public opinion analysis method provided in the first aspect are implemented.

[0015] The beneficial effect of the present invention is that the public opinion analysis method, system, device and storage medium provided by the present invention, combined with the BERT and LDA models, the system can more accurately extract topics from a large amount of text data, identify hot issues that tourists are concerned about, and help scenic area managers to respond quickly. At the same time, the BERT-BiLSTM-CNN model is used to analyze the emotional tendency of tourist satisfaction, so as to provide timely feedback on service improvement results and continuously optimize service quality. The present invention can collect data from a wide range of channels, and by first analyzing the public opinion theme and then analyzing the emotional tendency under each theme, it can realize hierarchical analysis of user evaluation data, providing data support for subsequent decision-making.

[0016] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0019] Figure 2 is another schematic flow chart of a method according to an embodiment of the present invention.

[0020] Figure 3 It is a schematic flow chart of identifying public opinion topics according to a method of an embodiment of the present invention.

[0021] Figure 4 is a schematic block diagram of a system according to an embodiment of the present invention.

[0022] Figure 5 A schematic diagram of the structure of a device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0025] The public opinion analysis method provided in the embodiment of the present invention is executed by a computer device, and accordingly, the public opinion analysis system runs in the computer device.

[0026] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention. Figure 1 The execution subject may be a public opinion analysis system. According to different requirements, the order of the steps in the flowchart may be changed, and some may be omitted.

[0027] like Figure 1 As shown, the method includes: S1. Collect user evaluation data and pre-process the user evaluation data.

[0028] Collect data: Collect user evaluation data from a wide range of channels, including social media, online forums, user reviews, news reports, etc. These data should cover different time periods and diverse user groups to ensure the comprehensiveness and accuracy of the analysis.

[0029] Data preprocessing: Clean and organize the collected data, including removing irrelevant information (such as advertisements, spam comments, etc.), word segmentation, removing stop words, stemming or word form restoration, etc. These preprocessing operations help reduce data noise and improve the efficiency and accuracy of subsequent analysis.

[0030] S2. Use pre-trained language models and topic models to analyze public opinion topics of pre-processed user evaluation data.

[0031] Pre-trained language model: Use pre-trained language models (such as BERT, GPT, etc.) to encode the processed text to capture the semantic information of the text. These models are trained on a large amount of text and can understand the complex context and semantic relationships in the text, providing a basis for subsequent topic analysis.

[0032] Topic model: Based on the pre-trained language model, a topic model (such as LDA, NMF, etc.) is applied to identify hidden topics in the text. The topic model analyzes the co-occurrence relationship of words in the text and clusters similar words together to form different topics. By adjusting the parameters of the topic model (such as the number of topics, number of iterations, etc.), more accurate topic identification results can be obtained.

[0033] S3. Group the preprocessed user evaluation data according to the public opinion topics to which they belong, to obtain multiple data groups.

[0034] Topic label assignment: Based on the analysis results of the topic model, one or more topic labels are assigned to each user evaluation data. This can be achieved by calculating the similarity between the text and each topic, and selecting the topic with the highest similarity as the label of the text.

[0035] Data grouping: User evaluation data is grouped according to the assigned topic labels. Each data group represents a specific public opinion topic and contains all user evaluations related to that topic. The grouped data can be used for subsequent tasks such as sentiment analysis and keyword extraction to further gain a deeper understanding of user feedback and opinions on specific topics.

[0036] S4. Use a sentiment analysis model to respectively identify the sentiment tendencies of the multiple data groups to obtain the sentiment tendencies corresponding to the multiple public opinion topics, wherein the sentiment analysis model includes a pre-trained language model, a bidirectional long short-term memory network model and a convolutional neural model.

[0037] Sentiment analysis model: Advanced sentiment analysis technologies such as pre-trained language models, bidirectional long short-term memory network (BiLSTM) models, and convolutional neural network (CNN) models are used to identify the sentiment tendency of grouped user evaluation data. These models can capture the sentiment characteristics in the text and accurately determine the sentiment tendency of the text (such as positive, negative, neutral, etc.).

[0038] Sentiment tendency analysis: Perform sentiment tendency analysis on the user evaluation data in each data group to obtain the sentiment tendency corresponding to the data group. This helps to understand the user's overall emotional attitude and emotional changes towards a specific public opinion topic.

[0039] Please refer to Figure 2 Through a series of steps such as collecting user evaluation data, preprocessing, topic analysis, data grouping and sentiment analysis, we can fully and deeply understand the public's views and sentiments on different public opinion topics. This helps companies to grasp the direction of public opinion in a timely manner and formulate more scientific and reasonable decisions and response strategies.

[0040] In an embodiment of the present invention, based on step S1, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0041] S101. Data Collection Use crawler technology to crawl user evaluation data of target objects from multiple platforms: First, determine the target object that needs to be crawled, such as a brand, product, or service.

[0042] According to the target object, select relevant social media platforms, online forums, e-commerce platforms, etc. as data sources.

[0043] Use crawler technology (such as Python's Scrapy, BeautifulSoup and other libraries) to write crawler scripts to crawl user evaluation data of the target object from the selected platform. The crawler script should be able to handle the platform's anti-crawler mechanism, such as verification code verification, IP blocking, etc.

[0044] During the crawling process, record key information such as the source, time, and evaluation content of the data.

[0045] Save the crawled user evaluation data to the folder pre-built for the corresponding platform: Create a separate folder for each platform to store the user review data crawled from that platform.

[0046] Folders can be further subdivided according to the source and time of the data to facilitate subsequent data management and analysis.

[0047] Use appropriate file formats (such as CSV, JSON, etc.) to save data to ensure that the data is readable and processable.

[0048] S102. Data cleaning Clean the user evaluation data and remove invalid fields and escape characters in the user evaluation data: Traverse all user review data, check and remove invalid fields, such as empty fields, duplicate fields, etc.

[0049] Identify and remove escape characters in data, such as HTML tags and special characters, to ensure data purity.

[0050] Standardize the data, such as unifying the date format and removing extra spaces, to improve the accuracy of subsequent analysis.

[0051] S103. Data preprocessing Use the word segmentation tool and custom dictionary to segment the sentences in the cleaned user evaluation data to obtain the word sequence: Select a suitable word segmentation tool (such as jieba, NLTK, etc.) to perform word segmentation on the cleaned user evaluation data.

[0052] Based on your analysis needs, build custom dictionaries to accurately identify and analyze domain-specific terms and vocabulary.

[0053] Use word segmentation tools and custom dictionaries to segment sentences and obtain word sequences, which is helpful for subsequent tasks such as sentiment analysis and topic identification.

[0054] Post-process the word segmentation results, such as removing stop words, performing stem extraction or word form restoration, to further reduce data noise and improve the accuracy of analysis.

[0055] In a specific example, in order to obtain the public's views on scenic spots as comprehensively as possible, data collection must be extensive enough. This system is mainly based on web crawler technology, using web crawler frameworks such as Selenium and Scrapy to traverse web page related data and store the collected data on the server.

[0056] After crawling the data, subsequent preprocessing is crucial for the implementation of downstream tasks. The quality of the data set greatly affects the accuracy of subsequent analysis work. Therefore, after obtaining the text data, it must undergo a series of processing to filter out the messy content and remove the noise data of low importance to achieve a standardized representation.

[0057] Due to the colloquial nature of comment texts, we first need to clean some common symbols and insignificant fields, escape characters, etc. in the collected data and convert them into more standardized Chinese text. Secondly, it is necessary to perform word segmentation on the text data, mainly based on the Jieba word segmentation tool, while adding some custom dictionary content that fits the cultural and tourism industry to reasonably segment the sentences. After that, it is necessary to remove stop words to simplify the text. At the same time, in response to the increasingly diversified expressions on social platforms, some emoticons, Emoji expressions, and Internet terms all play an important role in expressing emotions. Therefore, in the data preprocessing stage, we will also refer to the National Language Resource Monitoring Corpus to construct a conversion table for popular Internet phrase abbreviations, and replace these unconventional expressions with standard text languages.

[0058] In an embodiment of the present invention, based on step S2, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0059] S201. Word vector generation Extract the word sequence of user review data in the target folder: Read and extract the word sequence after preprocessing (such as word segmentation and stop word removal) from the target folder where the user evaluation data was previously stored.

[0060] Ensure the completeness and accuracy of the word sequence and provide a reliable data foundation for subsequent word vector generation.

[0061] Input the word sequence into the pre-trained language model to obtain the corresponding word vector set: The BERT model was chosen, which has been trained on a large-scale corpus and is able to capture the semantic relationships between words.

[0062] The extracted word sequence is input into the pre-trained language model, and the embedding layer of the model is used to convert each word into a word vector in a high-dimensional space.

[0063] Summarize the word vectors of all words to form a word vector set for subsequent analysis and processing.

[0064] S202. Word vector deduplication Calculate the similarity between word vectors in a word vector set: Using cosine similarity, calculate the similarity between each pair of word vectors in the word vector set.

[0065] The higher the similarity value, the closer the two word vectors are semantically.

[0066] Mark word vectors whose similarity exceeds the set similarity threshold as duplicate word vectors: Set a reasonable similarity threshold based on analysis requirements.

[0067] For word vector pairs whose similarity exceeds the threshold, one (or both) of them is marked as a duplicate word vector.

[0068] The existence of duplicate word vectors may affect the subsequent topic identification results, so deduplication processing is required.

[0069] De-duplicate the word vector set to obtain a streamlined word vector set: Traverse the word vector set and delete the word vectors marked as duplicates.

[0070] Unique word vectors are retained to form a streamlined word vector set to reduce data redundancy and improve analysis efficiency.

[0071] S203. Subject Identification The simplified word vector set is input into the topic model to obtain the public opinion topic: Choose an appropriate topic model, LDA, which can identify topics in a text based on the distribution of word vectors.

[0072] The streamlined word vector set is input into the topic model, and the model's algorithm and mechanism are used to identify public opinion topics.

[0073] Public opinion topics are usually presented in the form of keywords or phrases, reflecting the core content and focus of user evaluation data.

[0074] And the distribution of public opinion topics in the text: In addition to identifying public opinion topics, topic models can also provide distribution information of public opinion topics in text.

[0075] This includes how often each topic appears in the text, where it appears, and how it is related to other topics.

[0076] By analyzing the distribution information of public opinion topics, we can further understand the users’ attention level and attitude changes towards different topics.

[0077] In a specific example, since comment sentences are often short and incoherent, LDA may not be able to extract co-occurring words well, affecting the effect of topic generation; therefore, the system uses BERT as a text vectorization representation tool to model the text and convert it into word vectors. , use similarity to extract text features, and after filtering out texts with similar meanings, integrate the input into the LDA model to calculate the subject words and description words.

[0078] The LDA model is a generative model that learns probability distributions from a document-word matrix. Gibbs sampling is used to infer the distribution of each topic:

[0079] LDA can automatically discover different topics and determine their distribution in the text. After the model training is completed, the word distribution of each topic and the topic distribution in the text can be obtained.

[0080] In the expression, It is a hyperparameter of the Dirichlet distribution in the document, which is used to generate the probability distribution of document topics. It is a hyperparameter that measures the distribution of topic-related words and obeys the Dirichlet distribution, and is used to generate N topic distributions.

[0081] Based on the topic distribution learned by the LDA model, the meaning of each topic can be explained. A topic may contain some related words, through which the meaning of the topic can be understood. In addition, the topic distribution of each text can be seen to determine which topics are contained in the text.

[0082] The whole process is as follows Figure 3 shown.

[0083] Further explanation of the fusion model: Application of BERT model: Text modeling and word vector generation: Use the BERT model to model the text and convert it into a word vector in a high-dimensional space. Through a large amount of pre-training, the BERT model can capture the semantic relationship between words, providing a basis for subsequent similarity calculation and topic identification.

[0084] Similarity calculation and text feature extraction: Calculate the similarity between word vectors to identify texts with similar meanings. This helps filter out repeated or redundant information and improve the accuracy of topic identification. Through similarity calculation, representative text features are extracted for subsequent LDA model input.

[0085] Applications of LDA Model: Introduction to LDA model: LDA is a generative model that can learn probability distribution from the document-word matrix, automatically discover different topics, and determine their distribution in the text. The training process of the LDA model usually uses algorithms such as Gibbs sampling to infer the distribution of each topic.

[0086] LDA model training: The text features processed and filtered by BERT are integrated and input into the LDA model. By training the LDA model, the word distribution of each topic and the topic distribution in the text are obtained. In the LDA model, the hyperparameters α and β are used to generate the document topic probability distribution and the distribution of topic-related words, respectively, which follow the Dirichlet distribution.

[0087] Topic interpretation and distribution analysis: Based on the topic distribution learned by the LDA model, the meaning of each topic can be explained. Each topic may contain some related words that can reflect the core content of the topic. By analyzing the topic distribution of the text, it is possible to determine which topics are contained in the text and the relative importance of these topics.

[0088] By combining the application of BERT and LDA models, it is possible to more effectively process short and incoherent comment sentences and improve the accuracy and reliability of topic identification, which provides strong support for subsequent public opinion analysis and decision-making.

[0089] In an embodiment of the present invention, based on step S3, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0090] S301. Theme naming and explanation After processing text data using topic models such as LDA, we will get the vocabulary distribution of each topic. These vocabulary distributions reflect the core content and characteristics of the topic.

[0091] Topic naming: Based on the vocabulary distribution of each topic, we can name it. When naming, try to choose words or phrases that can summarize the core content of the topic to make it representative and easy to understand.

[0092] For example, if the words “smart,” “mobile phone,” and “function” frequently appear in the vocabulary distribution of a topic, we can name it “smartphone function topic.”

[0093] Topic explanation: In addition to naming, the topic also needs to be explained. When explaining, you can list some key words in the topic and briefly explain the connection between them and the core content of the topic.

[0094] For example, for the “smartphone function theme”, we can interpret it as focusing on the various functions of smartphones, including photography, games, battery life, etc., and involving users’ evaluation and feedback on these functions.

[0095] S302. Topic tag generation Based on the naming and explanation of the topics, we can generate corresponding topic tags for each topic. Topic tags are short representations of topics and are usually used for classification and retrieval.

[0096] Label design: The label should be concise, easy to understand and remember. Keywords, phrases or abbreviations can be used as labels.

[0097] For example, for the “smartphone features topic”, we could use “smartphone features” as the topic tag.

[0098] Label assignment: Assign the generated topic labels to the corresponding topics for use in subsequent analysis and processing.

[0099] S303. User evaluation data topic label assignment After obtaining the topics and topic labels, we need to associate the user evaluation data with the topics and assign one or more topic labels to each user evaluation data.

[0100] Similarity calculation: Use appropriate similarity calculation methods (such as cosine similarity, Jaccard similarity, etc.) to calculate the similarity between each user evaluation data and each topic. The higher the similarity value, the stronger the correlation between the user evaluation data and the topic.

[0101] Label assignment: One or more topic labels are assigned to each user evaluation data based on the similarity calculation results. A similarity threshold can be set, and only when the similarity between the user evaluation data and the topic exceeds the threshold, the corresponding topic label is assigned to it. It is also possible to consider assigning multiple topic labels to the user evaluation data to reflect the multiple topics it may involve.

[0102] S304. Data Group Division Finally, user evaluation data with the same topic label are divided into the same data group.

[0103] Data group creation: Create a data group for each topic label. User evaluation data in a data group should have the same topic label.

[0104] Data division: Traverse all user evaluation data and divide them into corresponding data groups according to their topic tags. This makes it easier to conduct centralized analysis and processing of user evaluation data on the same topic in the future.

[0105] Through the above process, text topic identification, label assignment and data group division can be achieved, providing strong support for subsequent user evaluation data analysis and public opinion monitoring.

[0106] In an embodiment of the present invention, based on step S4, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0107] The sentiment analysis model is the BERT-BiLSTM-CNN model. The model includes: (1) Pre-trained language model: Pre-trained language model is an important tool in the field of natural language processing. It can learn the statistical laws of language from large amounts of text data and provide powerful representation capabilities for various downstream tasks.

[0108] Model structure: Input layer: Three layers of vectorization preprocessing are used: word vector, text vector and position vector. Word vector represents the semantic information of words, text vector represents the overall information of text, and position vector is used to capture the position information of words in text. The combination of these three layers of vectors can more comprehensively represent text data.

[0109] Encoding layer: It is formed by accumulating 24 bidirectional Transformer encoders. The internal structure of each Transformer encoder includes 1024 hidden layers and 16 attention mechanisms, which enables the model to capture long-distance dependencies and contextual information in the text.

[0110] Output layer: Outputs a word vector matrix, which contains the vector representation of each word in the text and provides input for subsequent tasks.

[0111] Fine-tuning training: The pre-trained language model is fine-tuned using the Masked Language Model (MLM) and context prediction tasks. The MLM task randomly masks some words in the text and requires the model to predict the masked words based on the context, thereby enhancing the model's ability to understand language.

[0112] The context prediction task requires the model to predict the context based on a given text fragment, further improving the model's semantic understanding ability.

[0113] It is understandable that the common Word2vec embedding method is often limited in understanding the context due to the limited scope of information extraction. Therefore, the system uses the BERT model for vectorization preprocessing in the satisfaction sentiment analysis module. The BERT structure includes an input layer, an encoding layer, and an output layer. The BERT input layer uses three layers of vectorization preprocessing for text input: token embedding, segment embedding, and position embedding. First, "[CLS]" and "[SEP]" are added to the start and end positions of the text for marking. Segment embedding generates text global semantic variables and position embedding performs position encoding vector conversion. Then, the three vector matrices are added to obtain the input variables of the BERT model. The BERT encoding layer is the core module. The system uses the BERT-Large model, which is formed by the accumulation of 24 bidirectional Transformer encoders. The internal structure includes 1024 hidden layers and 16 attention mechanisms. It is then combined with the Masked language model task and context prediction task to perform model fine-tuning training to adapt to the sentiment analysis task of tourist attraction public opinion text data.

[0114] (2) Bidirectional Long Short-Term Memory (Bi-LSTM) is a special recurrent neural network (RNN) that can capture bidirectional information in text.

[0115] Bidirectional encoding mode: Bi-LSTM adopts a bidirectional encoding mode, which considers both the forward and reverse information of the text at the same time. This enables the model to capture the contextual dependencies in the text more accurately.

[0116] Eigenvector capture: Bi-LSTM takes the word embedding matrix output by the pre-trained language model as input and captures the feature vectors in the text through its internal structure. These feature vectors contain key information in the text, such as the semantics of the vocabulary and the syntactic structure.

[0117] Output feature concatenation: The captured feature vectors are concatenated to form output features, which will be used for subsequent task processing.

[0118] It is understandable that after BERT processing is completed, the result is used as the input of the BiLSTM-CNN model for feature extraction. Since the review text is often short, the sentiment polarity of the sentence is often concentrated on a certain word. The CNN model focuses on local features to help the experiment achieve good classification results, but CNN has major defects in processing long texts such as transitional sentences. BiLSTM can learn the changes in text semantics by using a bidirectional model. Therefore, the system combines the two models to extract semantic features from both the local and overall levels to obtain more accurate satisfaction sentiment analysis.

[0119] (3) Convolutional Neural Network (CNN) is mainly used in text processing for tasks such as local feature recognition and sentiment classification.

[0120] Local feature recognition: CNN performs local feature recognition on the features output by Bi-LSTM. The convolution kernel slides on the features to extract the association information between adjacent words. This information helps the model understand the local structure and semantic relationship in the text.

[0121] Max Pooling Layer: Apply the max pooling layer to update and reduce the key words. The max pooling layer can retain the maximum value in each convolution kernel, thereby filtering out the most important feature information.

[0122] Fully connected layer and sentiment polarity classification: The features output by CNN are input into the fully connected layer for nonlinear processing, which helps the model learn more complex feature combinations and relationships.

[0123] Finally, the softmax classifier is used to classify the sentiment polarity of the key words after nonlinear processing. The softmax classifier can map features to the probability distribution of sentiment polarity, thereby realizing the judgment of the sentiment tendency of the text.

[0124] It is understandable that the functions and mechanisms of the BiLSTM model include: Bidirectional encoding mode: The BiLSTM model can capture both the forward and reverse information of the text through the bidirectional encoding mode. This means that the model not only considers the previous information of the current word, but also the subsequent information, thus more comprehensively understanding the context of the text.

[0125] Input and feature extraction: The input of the BiLSTM model is usually text data that has been preprocessed (such as word segmentation, stop word removal, etc.). These data are first converted into a word vector matrix. Each row in the word vector matrix represents the vector representation of a word, and these vectors contain the semantic information of the word.

[0126] In the BiLSTM model, there are two layers of LSTM (Long Short-Term Memory) structures, which process the forward and reverse information of the text respectively. Each layer of LSTM extracts feature vectors from the text, which contain the contextual information of the text at different positions.

[0127] Feature vector concatenation: In order to combine the forward and backward feature vectors, the model concatenates the feature vectors extracted by the two layers of LSTM. In this way, the final feature vector of each word contains complete information about its context.

[0128] The role and mechanism of CNN model: Local feature recognition: The CNN model is good at capturing the local features of the text, that is, the association between adjacent words. It slides the convolution kernel on the feature vector to extract the local feature information in the text.

[0129] Input and feature extraction: The input of the CNN model is the feature vector matrix extracted by the BiLSTM model. These feature vectors already contain the contextual information of the text, so the CNN model can further extract local features based on this.

[0130] Max pooling layer: In order to reduce the dimension of the data and retain the most important feature information, CNN models usually apply a max pooling layer. The max pooling layer selects the maximum value in each convolution kernel as the output of the convolution kernel, thereby updating and reducing the data.

[0131] Fully connected layer and softmax classifier: Fully connected layer: After being processed by the CNN model, the feature vector is sent to the fully connected layer. Each neuron in the fully connected layer is connected to all neurons in the previous layer for further processing of the feature vector.

[0132] Activation function tanh: In the fully connected layer, an activation function is usually used to increase the nonlinear ability of the model. The tanh activation function is a common choice, which can map the feature vector to the range of -1 to 1, thereby enhancing the expressive power of the model.

[0133] Softmax classifier: Use the softmax classifier to classify the feature vector into sentiment polarity. The softmax classifier can map the feature vector to the probability distribution of sentiment polarity, thereby realizing the judgment of the sentiment tendency of the text.

[0134] In tourism scenarios, sentiment analysis models can be widely used to analyze tourists' evaluations and feedback on attractions, hotels, restaurants, etc. The following is a specific example that shows how to use sentiment analysis models to perform sentiment analysis on all data groups, and to calculate sentiment tendencies based on sentiment analysis results, and then establish a corresponding relationship between public opinion topics and sentiment tendencies.

[0135] After conducting sentiment analysis on the data groups corresponding to each public opinion topic, the following processing method is also used for subsequent processing.

[0136] 1. Emotional tendency statistics and public opinion theme analysis Sentiment trend statistics: The sentiment trend of all review data is counted to obtain the number and proportion of positive, negative and neutral reviews. This helps to understand the overall sentiment trend of tourists towards the attractions.

[0137] Public opinion topic extraction: Extract the public opinion topics that tourists are concerned about from the comment data, such as scenic spot environment, service quality, catering prices, etc. This can be achieved through methods such as keyword extraction and text clustering.

[0138] Establishing a corresponding relationship: Based on the corresponding relationship between the data group to which the sentiment tendency belongs and the public opinion topic, establish the corresponding relationship between the public opinion topic and the sentiment tendency. For example, the number and proportion of positive, negative, and neutral comments under each public opinion topic can be counted to understand the tourists' sentiment tendency towards different public opinion topics.

[0139] 2. Result analysis and application Result analysis: Based on the statistical results and corresponding relationships, analyze the tourists’ emotional tendencies and public opinion topics towards the scenic spots. For example, if it is found that negative comments are mainly focused on service quality, the scenic spot management can focus on and improve the service quality.

[0140] Application suggestions: Based on the analysis results, make targeted application suggestions. For example, strengthen employee training, optimize service processes, improve catering quality, etc. At the same time, the analysis results can also be used in scenic spot publicity, market promotion, etc. to attract more tourists.

[0141] In some embodiments, the public opinion analysis system may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the public opinion analysis system may be stored in the memory of a computer device and executed by at least one processor to perform (see Figure 1 Description) The function of public opinion analysis.

[0142] In this embodiment, the public opinion analysis system can be divided into multiple functional modules according to the functions it performs, such as Figure 4As shown. The functional modules of the system 400 may include: a data collection module 410, a topic analysis module 420, a data grouping module 430 and a sentiment analysis module 440. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0143] A data collection module, used to collect user evaluation data and pre-process the user evaluation data; The topic analysis module is used to analyze the public opinion topics of the pre-processed user evaluation data using the pre-trained language model and the topic model; A data grouping module is used to group the pre-processed user evaluation data according to the public opinion topic to obtain multiple data groups; The sentiment analysis module is used to use the sentiment analysis model to respectively identify the sentiment tendencies of the multiple data groups and obtain the sentiment tendencies corresponding to the multiple public opinion topics. The sentiment analysis model includes a pre-trained language model, a bidirectional long short-term memory network model and a convolutional neural model.

[0144] Optionally, as an embodiment of the present invention, collecting user evaluation data and preprocessing the user evaluation data includes: Use crawler technology to crawl user evaluation data of target objects from multiple platforms, and save the crawled user evaluation data into a folder pre-built for the corresponding platform; Cleaning the user evaluation data to remove invalid fields and escape characters in the user evaluation data; Use word segmentation tools and custom dictionaries to segment sentences in the cleaned user evaluation data to obtain word sequences.

[0145] Optionally, as an embodiment of the present invention, using a pre-trained language model and a topic model to analyze the public opinion topics of the pre-processed user evaluation data includes: Extracting a word sequence of user evaluation data in a target folder, inputting the word sequence into a pre-trained language model, and obtaining a corresponding word vector set; Calculate the similarity between word vectors in the word vector set, mark the word vectors whose similarity exceeds the set similarity threshold as duplicate word vectors, and perform deduplication processing on the word vector set to obtain a streamlined word vector set; The simplified word vector set is input into the topic model to obtain the public opinion topic and the distribution of the public opinion topic in the text.

[0146] Optionally, as an embodiment of the present invention, the pre-processed user evaluation data is grouped according to the public opinion topic to which it belongs, to obtain multiple data groups, including: According to the vocabulary distribution of each topic in the topic model, the topic is named and explained, and the corresponding topic label is generated for the topic based on the naming and explanation of the topic; By calculating the similarity between each user evaluation data and each topic, one or more topic labels are assigned to each user evaluation data; User evaluation data with the same topic label are divided into the same data group.

[0147] Optionally, as an embodiment of the present invention, the sentiment analysis model includes: A pre-trained language model includes an input layer, an encoding layer and an output layer. The input layer is pre-processed by three-layer vectorization of word vector, text vector and position vector. The encoding layer is formed by accumulating 24 bidirectional Transformer encoders. The internal structure includes 1024 hidden layers and 16 attention mechanisms. The output layer outputs a word vector matrix. The pre-trained language model is fine-tuned using a masked language model and a context prediction task. A bidirectional long short-term memory network model, wherein the bidirectional long short-term memory network model uses a bidirectional encoding mode to capture feature vectors of the word vector matrix, and concatenates the captured feature vectors into output features; A convolutional neural model is provided, wherein the convolutional neural model performs local feature recognition on the output features, obtains the association between adjacent words, applies a maximum pooling layer to update and reduce key words, outputs the final key words to a fully connected layer, uses an activation function to perform nonlinear processing on the key words, and uses a softmax classifier to perform sentiment polarity classification on the key words after the nonlinear processing.

[0148] Optionally, as an embodiment of the present invention, it further includes: Use sentiment analysis models to conduct sentiment analysis on all data groups, and calculate sentiment tendencies based on sentiment analysis results; Based on the correspondence between the data group to which the sentiment tendency belongs and the public opinion topic, a correspondence between the public opinion topic and the sentiment tendency is established.

[0149] Figure 5The public opinion analysis method provided for the embodiment of the present application can be applied to the device. Those skilled in the art will understand that the device structure involved in the embodiment of the present invention does not constitute a limitation on the device, and the device may include more or fewer components than shown, or combine certain components, or arrange different components. In an embodiment of the present invention, the device includes but is not limited to a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or required herein.

[0150] The device 500 may include: a processor 510, a memory 520 and a communication unit 530. These components communicate via one or more buses. Those skilled in the art will appreciate that the server structure shown in the figure does not limit the present invention, and it may be a bus structure or a star structure, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0151] The memory 520 may be used to store the execution instructions of the processor 510, and the memory 520 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 520 are executed by the processor 510, the device 500 is enabled to perform some or all of the steps in the following method embodiments.

[0152] The processor 510 is the control center of the storage device, and uses various interfaces and lines to connect various parts of the entire electronic device. It runs or executes software programs and / or modules stored in the memory 520, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of a plurality of packaged ICs with the same or different functions. For example, the processor 510 can include only a central processing unit (CPU). In an embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0153] The communication unit 530 is used to establish a communication channel so that the storage device can communicate with other devices, receive user data sent by other devices or send user data to other devices.

[0154] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0155] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes, including several instructions for enabling a computer device (which can be a personal computer, a server, or a second device, a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0156] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0157] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, which can be electrical, mechanical or other forms.

[0158] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0160] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A method for analyzing public opinion, characterized in that: include: Collecting user evaluation data and preprocessing the user evaluation data; Use pre-trained language models and topic models to analyze public opinion topics of pre-processed user evaluation data; The preprocessed user evaluation data is grouped according to the public opinion topics to which they belong, to obtain multiple data groups; The sentiment analysis model is used to identify the sentiment tendencies of the multiple data groups respectively, and the sentiment tendencies corresponding to the multiple public opinion topics are obtained. The sentiment analysis model includes a pre-trained language model, a bidirectional long short-term memory network model and a convolutional neural model.

2. The method according to claim 1, characterized in that Collecting user evaluation data and preprocessing the user evaluation data, including: Use crawler technology to crawl user evaluation data of target objects from multiple platforms, and save the crawled user evaluation data into a folder pre-built for the corresponding platform; Cleaning the user evaluation data to remove invalid fields and escape characters in the user evaluation data; Use word segmentation tools and custom dictionaries to segment sentences in the cleaned user evaluation data to obtain word sequences.

3. The method according to claim 2, characterized in that Use pre-trained language models and topic models to analyze the public opinion topics of pre-processed user evaluation data, including: Extracting a word sequence of user evaluation data in a target folder, inputting the word sequence into a pre-trained language model, and obtaining a corresponding word vector set; Calculate the similarity between word vectors in the word vector set, mark the word vectors whose similarity exceeds the set similarity threshold as duplicate word vectors, and perform deduplication processing on the word vector set to obtain a streamlined word vector set; The simplified word vector set is input into the topic model to obtain the public opinion topic and the distribution of the public opinion topic in the text.

4. The method according to claim 1, characterized in that: The pre-processed user evaluation data is grouped according to the public opinion topics to which it belongs, and multiple data groups are obtained, including: According to the vocabulary distribution of each topic in the topic model, the topic is named and explained, and the corresponding topic label is generated for the topic based on the naming and explanation of the topic; By calculating the similarity between each user evaluation data and each topic, one or more topic labels are assigned to each user evaluation data; User evaluation data with the same topic label are divided into the same data group.

5. The method according to claim 1, characterized in that The sentiment analysis model includes: A pre-trained language model includes an input layer, an encoding layer and an output layer. The input layer is pre-processed by three-layer vectorization of word vector, text vector and position vector. The encoding layer is formed by accumulating 24 bidirectional Transformer encoders. The internal structure includes 1024 hidden layers and 16 attention mechanisms. The output layer outputs a word vector matrix. The pre-trained language model is fine-tuned using a masked language model and a context prediction task. A bidirectional long short-term memory network model, wherein the bidirectional long short-term memory network model uses a bidirectional encoding mode to capture feature vectors of the word vector matrix, and concatenates the captured feature vectors into output features; A convolutional neural model is provided, wherein the convolutional neural model performs local feature recognition on the output features, obtains the association between adjacent words, applies a maximum pooling layer to update and reduce key words, outputs the final key words to a fully connected layer, uses an activation function to perform nonlinear processing on the key words, and uses a softmax classifier to perform sentiment polarity classification on the key words after the nonlinear processing.

6. The method according to claim 1, characterized in that The method further comprises: Use sentiment analysis models to conduct sentiment analysis on all data groups, and calculate sentiment tendencies based on sentiment analysis results; Based on the correspondence between the data group to which the sentiment tendency belongs and the public opinion topic, a correspondence between the public opinion topic and the sentiment tendency is established.

7. A public opinion analysis system, characterized in that: include: A data collection module, used to collect user evaluation data and pre-process the user evaluation data; The topic analysis module is used to analyze the public opinion topics of the pre-processed user evaluation data using the pre-trained language model and the topic model; A data grouping module is used to group the pre-processed user evaluation data according to the public opinion topic to obtain multiple data groups; The sentiment analysis module is used to use the sentiment analysis model to respectively identify the sentiment tendencies of the multiple data groups and obtain the sentiment tendencies corresponding to the multiple public opinion topics. The sentiment analysis model includes a pre-trained language model, a bidirectional long short-term memory network model and a convolutional neural model.

8. The system according to claim 7, characterized in that The data collection module comprises: A data crawling unit, used to crawl user evaluation data of target objects from multiple platforms using crawler technology, and save the crawled user evaluation data into a folder pre-built for the corresponding platform; A data cleaning unit, used to clean the user evaluation data and remove invalid fields and escape characters in the user evaluation data; The word segmentation processing unit is used to segment the sentences in the cleaned user evaluation data using the word segmentation tool and the custom dictionary to obtain word sequences.

9. A device, characterized in that: include: A memory, used for storing a public opinion analysis program; A processor, used to implement the steps of the public opinion analysis method as described in any one of claims 1-6 when executing the public opinion analysis program.

10. A computer-readable storage medium storing a computer program, characterized in that: The readable storage medium stores a public opinion analysis program, which, when executed by a processor, implements the steps of the public opinion analysis method according to any one of claims 1 to 6.

Images