Public opinion analysis method and device based on big data and deep learning and medium

Abstract

The invention relates to a big data and deep learning-based public opinion analysis method and apparatus, and a medium. The method comprises the steps of collecting text, image and video multi-modal public opinion data of multiple cities; performing cleaning, duplicate removal, word segmentation and stop word filtering preprocessing operation on the collected public opinion data; respectively extracting text features and image or video features by using a natural language processing technology and a computer vision technology, and fusing the features through a multi-modal fusion algorithm; by adopting a distributed system and a distributed task scheduling algorithm, cross-platform and cross-language efficient data acquisition is realized, resource allocation can be dynamically adjusted according to public opinion popularity of each city, the real-time performance and comprehensiveness of key city data are ensured, and meanwhile, through a method of combining a TF-IDF algorithm, an emotion dictionary and deep learning, the real-time performance of the city is improved. The data quality is effectively improved, and a reliable basis is provided for subsequent analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of public opinion analysis, and in particular to a public opinion analysis method, device and medium based on big data and deep learning. Background Art

[0002] In today's information society, public opinion analysis has become an important basis for decision-making by governments, enterprises and various organizations. With the rapid development of Internet technology, the multimodal public opinion data of text, images and videos generated on social media, news websites and forum platforms has exploded. These data contain rich public opinions, emotional tendencies and event development dynamics, which are of great significance for timely understanding of social public opinion, predicting public opinion trends and formulating response strategies.

[0003] Traditional public opinion analysis methods mainly rely on manual reading and statistics, which are not only inefficient but also difficult to process large-scale, multimodal data. With the rise of big data and deep learning technologies, new solutions have been provided for public opinion analysis. Big data technology can efficiently store, process and analyze massive data, while deep learning technology can automatically extract features from complex data to achieve high-precision prediction and classification.

[0004] However, the existing public opinion analysis methods based on big data and deep learning still have some shortcomings. First, in terms of data collection, how to efficiently obtain public opinion data in multiple cities, multiple platforms, and multiple languages ​​is still a challenge. Traditional systems often find it difficult to adapt to dynamically changing network environments and data formats, resulting in low data collection efficiency. Secondly, existing multimodal data fusion algorithms often find it difficult to fully capture the correlation between different modalities, resulting in inaccurate feature representation after fusion. Summary of the invention

[0005] The purpose of this invention is to provide a public opinion analysis method based on big data and deep learning.

[0006] In order to solve the above technical problems, the present invention provides a public opinion analysis method based on big data and deep learning, including:

[0007] Collect multi-modal public opinion data including text, images and videos from multiple cities;

[0008] Perform pre-processing operations such as cleaning, deduplication, word segmentation, and stop word filtering on the collected public opinion data;

[0009] Use natural language processing technology and computer vision technology to extract text features and image or video features respectively, and fuse these features through a multimodal fusion algorithm;

[0010] Based on the deep learning framework, we built a multi-city public opinion analysis model, and used historical public opinion data to train the model and optimize the model parameters;

[0011] Input the public opinion data collected in real time into the trained model to predict the development trend of public opinion in different cities and analyze the hot spots, propagation paths and impact range of public opinion events;

[0012] Based on the results of public opinion prediction and analysis, targeted public opinion response strategies are formulated for different cities, and the development of public opinion is monitored in real time and the response strategies are adjusted dynamically.

[0013] Furthermore, the collection of multi-modal public opinion data of text, image and video in multiple cities includes the following steps:

[0014] Public opinion data collection adopts a distributed system to achieve cross-platform and cross-language data capture, and dynamically adjusts resource allocation through distributed task scheduling. The distributed task scheduling method includes the following steps:

[0015] Use the public opinion monitoring system to analyze the public opinion heat, discussion volume and dissemination speed indicators of each city in real time, and dynamically adjust resources based on these data to allocate more resources to cities with more active public opinion;

[0016] A scheduling algorithm based on task priority is adopted to prioritize the public opinion monitoring tasks in cities with high public opinion popularity, ensuring the efficiency of data collection in key cities. The specific formula of the scheduling algorithm is:

[0017] P(t)=W d ×H(t)+W v ×V(t)+W s ×S(t),

[0018] P(t) is the priority of city t, H(t) is the public opinion heat of the city, V(t) is the discussion volume of the city, S(t) is the spread speed of the city, W d , W v , W s is the corresponding weight factor, and the optimal configuration of public opinion data collection resources in multiple cities is achieved through the scheduling algorithm.

[0019] Furthermore, the pre-processing operations of cleaning, de-duplication, word segmentation, and stop word filtering of the collected public opinion data include the following steps:

[0020] The TF-IDF algorithm is used for keyword extraction, and its calculation formula is as follows:

[0021]

[0022] TF(t,d) represents the frequency of term t in document d, DF(t) represents the number of documents containing term t, and N is the total number of documents;

[0023] For public opinion sentiment analysis, a method based on a combination of sentiment dictionary and deep learning is used, including:

[0024] Preliminary classification of positive and negative sentiments in the text using a predefined sentiment dictionary;

[0025] A recurrent neural network model is used to conduct an in-depth analysis of the sentiment tendency of public opinion data. The model input is text data that has been word vectorized, and the output is the public opinion sentiment tendency score.

[0026] Furthermore, natural language processing technology and computer vision technology are used to extract text features and image or video features respectively, and these features are fused through a multimodal fusion algorithm, including the following steps:

[0027] Text feature extraction: perform word vectorization on the preprocessed text data, use the word embedding model to obtain word vectors, and use the convolutional neural network to extract high-level text features;

[0028] Image or video feature extraction: For image data, use pre-trained convolutional neural networks to extract visual features; for video data, extract key frames and use 3D-CNN or LSTM models to obtain spatiotemporal features;

[0029] Multimodal feature fusion: Fusion of text features with visual features, using feature concatenation, weighted combination or attention mechanism, where the weighted combination formula is: F = αF1 + βF2, where F is the fused feature representation, F1 is the text feature, F2 is the visual feature, α and β are weight coefficients, satisfying α+β=1.

[0030] Furthermore, the public opinion analysis model is migrated from the source city to the target city through transfer learning technology, and the specific steps include:

[0031] Source city model training: Train the basic model on the source city with rich data to extract common model parameters, such as word embedding and convolution kernel parameters;

[0032] Adaptive fine-tuning of target cities: During the migration process, by identifying the differences between the public opinion characteristics of the target city and the source city, the model parameters are dynamically adjusted to improve the adaptability of the model in the new environment;

[0033] The loss function of transfer learning is: L = λ 1 L s +λ 2 L t , where L s , L tare the loss functions of the source city and the target city respectively, λ 1 ,λ 2 , is the balance factor.

[0034] Furthermore, the public opinion analysis model also includes a sentiment analysis model, which incorporates the geographical location and cultural difference information of the city into the model. The specific steps are:

[0035] The geocode of each city is used as one of the inputs of the model to capture the differences between cities;

[0036] Introducing culturally relevant feature embedding vectors, combined with the city’s historical public opinion data, language habits, and socioeconomic characteristics, to conduct a more fine-grained analysis of public opinion data;

[0037] When training the model, a multi-task learning framework is used to define multiple loss functions: L i is the loss function of the ith city, λ i is the weight factor for each city,

[0038] By gradually adjusting the weight of each city, the generalization ability of the model can be improved and the model's adaptability to differences in public opinion between cities can be enhanced.

[0039] Furthermore, the real-time collected public opinion data is input into the trained model to predict the development trend of public opinion in different cities, and the prediction of the time dimension of public opinion data based on the long short-term memory network model is also included, specifically including:

[0040] The time series analysis model is used to predict the time dimension of the public opinion data of each city. The model formula is: h t =f(W h h t-1 +W x x t +b), where h t is the hidden layer state at time t, x t is the input word vector of the current time step, W h , W x is the weight matrix, b is the bias;

[0041] By combining data on historical events, economic activities and cultural events in each city, a composite prediction model is constructed to predict public opinion hotspots, propagation paths and their scope of influence.

[0042] Furthermore, for the hot spots, development changes and propagation paths of public opinion in different cities, the system can automatically generate response strategies based on historical data and adjust the response plans in real time through reinforcement learning algorithms, including:

[0043] Automatically generate preliminary response strategies based on historical public opinion data and actual public opinion changes in different cities. The response strategies include public communication, information release, and policy adjustments to address different public opinions.

[0044] In the process of strategy generation, the city’s geographical location, cultural differences, and socio-economic characteristics are comprehensively considered;

[0045] The policy gradient-based reinforcement learning algorithm is used to construct an optimization model for public opinion response strategy. Its objective function is: J(θ) = Eπθ[R(τ)], where π θ is the strategy, R(τ) is the reward value of the public opinion response effect, and the public opinion control effect is optimized by adjusting the model parameters θ.

[0046] The second technical solution of the present invention is a method and device for analyzing public opinion based on big data and deep learning, and the device comprises:

[0047] Data collection module, used to collect multimodal public opinion data including text, images and videos from multiple cities;

[0048] The data preprocessing module is used to perform preprocessing operations such as cleaning, deduplication, word segmentation, and stop word filtering on the collected public opinion data;

[0049] The feature extraction and fusion module is used to extract text features and image or video features respectively using natural language processing technology and computer vision technology, and fuse these features through a multimodal fusion algorithm;

[0050] The model building module is used to build a multi-city public opinion analysis model based on a deep learning framework, and use historical public opinion data to train the model and optimize model parameters;

[0051] The public opinion prediction module is used to input the public opinion data collected in real time into the trained model, predict the development trend of public opinion in different cities, and analyze the hot spots, propagation paths, and impact range of public opinion events;

[0052] The strategy formulation module is used to formulate targeted public opinion response strategies for different cities based on the results of public opinion prediction and analysis, and to monitor the development of public opinion in real time and dynamically adjust the response strategies.

[0053] The third technical solution of the present invention is a readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the method described above.

[0054] The beneficial effects of the present invention are:

[0055] 1. By adopting a distributed system and distributed task scheduling algorithm, efficient data collection across platforms and languages ​​is achieved. Resource allocation can be dynamically adjusted according to the popularity of public opinion in each city to ensure the real-time and comprehensiveness of data in key cities. At the same time, through the combination of TF-IDF algorithm, sentiment dictionary and deep learning, the collected public opinion data is cleaned, deduplicated, segmented, and stop word filtered, which effectively improves the data quality and provides a reliable foundation for subsequent analysis.

[0056] 2. By utilizing natural language processing technology and computer vision technology, text features and image or video features are extracted respectively, and these features are fused through a multimodal fusion algorithm to achieve a comprehensive understanding of public opinion data. The multimodal fusion method can capture the correlation between different modalities, improve the accuracy and robustness of feature representation, and provide richer and more accurate information for public opinion analysis.

[0057] 3. Through transfer learning technology, the present invention migrates the public opinion analysis model from the data-rich source city to the target city, realizing the rapid adaptation of the model in the new environment. At the same time, by introducing information on the city’s geographical location and cultural differences and combining it with a multi-task learning framework, the generalization ability of the model and its adaptability to differences in public opinion between cities are further improved. In addition, the reinforcement learning algorithm is used to adjust and optimize the public opinion response strategy in real time, ensuring the effectiveness and pertinence of the response strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0059] Figure 1 is a stereogram of a preferred embodiment of a discharge head of the present invention;

[0060] Figure 2 It is a rear view of a preferred embodiment of the discharge head of the present invention. DETAILED DESCRIPTION

[0061] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0062] like Figure 1 As shown, the public opinion analysis method based on big data and deep learning of the present invention includes the following steps:

[0063] S1: Collecting text, image, and video multimodal public opinion data from multiple cities, wherein the collecting of text, image, and video multimodal public opinion data from multiple cities includes the following steps:

[0064] Public opinion data collection adopts a distributed system to achieve cross-platform and cross-language data capture, and dynamically adjusts resource allocation through distributed task scheduling. The distributed task scheduling method includes the following steps:

[0065] Use the public opinion monitoring system to analyze the public opinion heat, discussion volume, and dissemination speed indicators of each city in real time, and dynamically adjust resources based on these data to allocate more resources to cities with more active public opinion;

[0066] A scheduling algorithm based on task priority is used to prioritize the public opinion monitoring tasks in cities with high public opinion popularity, ensuring the efficiency of data collection in key cities. The specific formula of the scheduling algorithm is:

[0067] P(t)=W d ×H(t)+W v ×V(t)+W s ×S(t)

[0068] P(t) is the priority of city t, H(t) is the public opinion heat of the city, V(t) is the discussion volume of the city, S(t) is the spread speed of the city, W d , W v , W s is the corresponding weight factor, and the optimal configuration of public opinion data collection resources in multiple cities is achieved through the scheduling algorithm.

[0069] Cross-platform and cross-language crawling: The distributed system can crawl public opinion data on a variety of different platforms. Whether it is Chinese, English or multiple languages, it can be accurately collected to ensure coverage of diverse public opinions in different cities and cultural backgrounds. Distributed task scheduling: The distributed task scheduler dynamically allocates resources of each node to achieve efficient data collection and public opinion monitoring. In order to ensure the real-time and efficiency of data collection, a task priority mechanism is introduced in the scheduling strategy, with the aim of adopting differentiated collection strategies for public opinion data in different cities.

[0070] Priority calculation: Based on the public opinion indicators (heat, discussion volume, and propagation speed) of each city, the above formula is used to calculate the public opinion collection priority of each city. Cities with high priorities will be allocated resources first to ensure that the public opinion data of the city is collected in a timely and comprehensive manner. After calculating the priority of each city, resources are allocated to each city in order of priority. Cities with high priorities obtain more nodes, which can ensure detailed monitoring of the public opinion in the city and realize dynamic optimization configuration. This means that over time, the heat, discussion volume, and propagation speed of public opinion will change. The resource allocation can be dynamically adjusted by recalculating the priority, thereby maximizing the efficiency of data collection.

[0071] S2: Perform pre-processing operations such as cleaning, de-duplication, word segmentation, and stop word filtering on the collected public opinion data. The pre-processing operations such as cleaning, de-duplication, word segmentation, and stop word filtering on the collected public opinion data include the following steps: Using the TF-IDF (term frequency-inverse document frequency) algorithm to extract keywords, the calculation formula is as follows:

[0072]

[0073] TF(t,d) represents the frequency of term t in document d, DF(t) represents the number of documents containing term t, and N is the total number of documents.

[0074] For public opinion sentiment analysis, a method based on a combination of sentiment dictionary and deep learning is adopted, which specifically includes: preliminary classification of positive and negative sentiments in the text through a predefined sentiment dictionary; using a recurrent neural network model to conduct in-depth analysis of the sentiment tendencies of public opinion data. The model input is text data that has been word vectorized, and the output is the public opinion sentiment tendency score.

[0075] Construction of sentiment lexicon: Sentiment lexicon is a set of predefined vocabularies that are usually assigned positive or negative sentiment labels. For example, words such as "good", "beautiful", and "like" are classified as positive sentiment, while "bad", "angry", and "failure" are classified as negative sentiment. The construction of sentiment lexicon can be based on public datasets or manually formulated by domain experts to ensure that it contains unique vocabulary in the field.

[0076] Preliminary sentiment classification: First, match the text data with the sentiment dictionary, count the frequency of positive and negative words in the text, and preliminarily classify the sentiment tendency of the text: Positive sentiment: If the frequency of positive words is higher than that of negative words, the system will preliminarily judge the text as positive sentiment. Negative sentiment: If the frequency of negative words is higher than that of positive words, the system will preliminarily judge the text as negative sentiment. Neutral sentiment: If the frequencies of positive and negative words are similar or very few, they will be judged as neutral sentiment. Preliminary classification can quickly provide a basic judgment for subsequent sentiment analysis, and can also significantly reduce the computational pressure of deep learning models.

[0077] Although preliminary sentiment classification can give a rough classification of the sentiment of the text, it cannot handle complex contextual relationships or multiple meanings of language. Therefore, the present invention further introduces a recurrent neural network (RNN) model to perform deep sentiment analysis on public opinion data and word vectorization: the text needs to be converted into a machine-understandable form. Using the Word2Vec or GloVe model, each word in the text is converted into a vector form to retain the semantic and contextual information of the word. For example, after a given text is processed by word segmentation, the system will convert each word into a continuous vector. These vectors retain the semantic relationship between words, for example, the semantic relationship between "good" and "beautiful". The distance between word vectors is very close, while "bad" is far away. A recurrent neural network (RNN) is used to conduct in-depth analysis of text sentiment. Model input: text data that has been word-vectorized is input into the RNN model and enters the network in the form of a sequence. Each time step represents a word. Model output: The final output is a sentiment score (for example, from -1 to 1), which is used to measure the sentiment tendency of the text. Positive values ​​(such as +0.8): represent positive sentiment. The larger the score, the more positive the sentiment. Negative values ​​(such as -0.6): represent negative sentiment. The lower the score, the more negative the sentiment. Values ​​close to 0: represent text with neutral sentiment or difficult to judge sentiment.

[0078] Through noise filtering and deduplication, data quality is ensured, invalid information and duplicate data are removed, thereby improving the efficiency and accuracy of analysis. Through word segmentation and stop word filtering, unstructured natural language is converted into word sequences with semantic meaning, and useless information is eliminated to ensure that the data maintains valid information while reducing computational complexity. Through vectorization, the processed text and image data are converted into feature representations to provide input for subsequent deep learning and public opinion analysis models. Through this series of preprocessing operations, the data quality in the subsequent analysis process is ensured, making the model training and public opinion analysis more accurate and effective.

[0079] S3: Use natural language processing technology and computer vision technology to extract text features and image or video features respectively, and fuse these features through a multimodal fusion algorithm; Text feature extraction: Word vectorization is performed on the preprocessed text data, word vectors are obtained using a word embedding model, and high-level text features are extracted using a convolutional neural network. Image or video feature extraction: For image data, visual features are extracted using a pre-trained convolutional neural network; for video data, key frames are extracted and spatiotemporal features are obtained using a 3D-CNN or LSTM model. Multimodal feature fusion: Text features are fused with visual features using feature concatenation, weighted combination, or attention mechanism, where the weighted combination formula is: F=αF1+βF2, where F is the fused feature representation, F1 is the text feature, F2 is the visual feature, α and β are weight coefficients, satisfying α+β=1.

[0080] Through natural language processing technology and computer vision technology, the system can extract key features from text data and image or video data respectively. These features can capture the semantic information of the text and the visual content information. After completing the extraction of text features and visual features, a fusion algorithm is used to combine the features of the two modalities. The feature fusion process achieves effective integration of different information sources through feature splicing, weighted combination or attention mechanism, thereby improving the overall understanding of public opinion. The fused feature representation contains comprehensive information of text and vision, which can be input into the public opinion analysis model to perform public opinion sentiment analysis, hot event identification and prediction of public opinion propagation paths. Multimodal feature fusion enables the model to understand public opinion from multiple perspectives with higher accuracy and robustness.

[0081] S4: Input the public opinion data collected in real time into the trained model, predict the development trend of public opinion in different cities, and analyze the hot spots, propagation paths, and impact range of public opinion events; Text feature extraction: Word vectorization is performed on the preprocessed text data, and the word vector is obtained using the word embedding model. The high-level text features are extracted using the convolutional neural network. Image or video feature extraction: For image data, the visual features are extracted using the pretrained convolutional neural network; for video data, the key frames are extracted and the spatiotemporal features are obtained using the 3D-CNN or LSTM model. Multimodal feature fusion: The text features are fused with the visual features, and feature splicing, weighted combination or attention mechanism is used. The formula for weighted combination is: F=αF1+βF2, where F is the fused feature representation, F1 is the text feature, F2 is the visual feature, α and β are weight coefficients, satisfying α+β=1. The weighted combination method can flexibly control the influence of different modalities by adjusting the weight factor to adapt to the analysis needs in different public opinion scenarios.

[0082] S41: The public opinion analysis model is migrated from the source city to the target city through transfer learning technology. The specific steps include: source city model training: training the basic model on the source city with rich data to extract common model parameters, such as vocabulary embedding and convolution kernel parameters; adaptive fine-tuning of the target city: during the migration process, by identifying the differences between the public opinion characteristics of the target city and the source city, the model parameters are dynamically adjusted to improve the adaptability of the model in the new environment; the loss function of transfer learning is: L = λ 1 L s +λ 2 L t Among them, L s , L t are the loss functions of the source city and the target city respectively, λ 1 ,λ 2 , is the balance factor.

[0083] First, the model is pre-trained on the data of the source city to extract common features related to public opinion. The basic model trained in this way has strong generalization ability and is suitable for migration to other cities. After the model is migrated to the target city, the data of the target city is analyzed to identify the differences between it and the source city in terms of language and public opinion characteristics, and adaptive adjustments are made. For example, the vocabulary embedding is updated to adapt to the language habits of the target city. In the process of fine-tuning the model in the target city, the model parameters are optimized through the joint loss function. On the basis of maintaining the learning results of the source city, the model can gradually adapt to the data characteristics of the target city. Through transfer learning, the public opinion analysis model can effectively adapt to the characteristics of different cities and improve the sensitivity and accuracy of changes in public opinion in the target city. Since the data of the target city may be relatively small, through transfer learning, the rich data of the source city can be used to pre-train the model, thereby reducing the demand for labeled data in the target city and saving the cost of data acquisition and labeling. The use of the joint loss function ensures that the model can not only retain the learning results in the source city, but also effectively adapt to the new characteristics of the target city.

[0084] S42: The public opinion analysis model also includes a sentiment analysis model, which integrates the geographical location and cultural difference information of the city into the model. The specific steps are: taking the geographical code of each city as one of the inputs of the model to capture the differences between cities; introducing culture-related feature embedding vectors, combining the historical public opinion data, language habits and socio-economic characteristics of the city, and performing a more fine-grained analysis of the public opinion data; when training the model, using a multi-task learning framework, defining multiple loss functions: L i is the loss function of the ith city, λ i The weight factor for each city is used. By gradually adjusting the weight of each city, the generalization ability of the model is improved and the model's adaptability to differences in public opinion between cities is enhanced.

[0085] By taking the city's geographic coding and cultural-related features as input features, the model not only relies on the semantic information of the text itself when performing sentiment analysis, but also combines the geographic location and cultural background to perform more personalized sentiment analysis. Under the multi-task learning framework, the system will treat the data of each city as an independent task and use multiple loss functions for training. By continuously adjusting the weight of each loss function, the model gradually learns how to process data from multiple cities at the same time to maximize the overall generalization ability. Through the introduction of geographic coding and cultural features, combined with the training method of multi-task learning, the model can more flexibly adapt to the different public opinion characteristics of each city, especially for those target cities that are quite different from the source cities. The model can ensure that the data of these cities are also fully learned and adapted through the training process of high weight factors and independent tasks.

[0086] S5: Input the public opinion data collected in real time into the trained model, predict the development trend of public opinion in different cities, and analyze the hot spots, propagation paths, and impact range of public opinion events; by integrating geographical location and cultural difference information into the sentiment analysis model, the geographic code and cultural characteristics of each city are used as model inputs. Combined with the multi-task learning framework, the generalization ability of the model is enhanced by defining and optimizing multiple loss functions, and the weight factors are gradually adjusted to cope with the unique characteristic differences of each city. This can enable the model to better adapt to different public opinion environments and enhance the sentiment analysis capabilities of each city.

[0087] The prediction of the development trend of public opinion in different cities also includes the prediction of the time dimension of public opinion data based on the long short-term memory network (LSTM) model, which specifically includes: predicting the time dimension of public opinion data of each city through a time series analysis model (such as the long short-term memory network LSTM). The model formula is: h t =f(W h h t-1 +W x x t +b) where h t is the hidden layer state at time t, x t is the input word vector of the current time step, W h , W x is the weight matrix, b is the bias, and the historical events, economic activities, and cultural events data of each city are combined to build a composite prediction model to predict public opinion hotspots, propagation paths, and their scope of influence. During the training process, the system will gradually adjust the weight factor of each city to improve the generalization ability of the model. This means that for cities with large differences in public opinion characteristics, the weight factor may be appropriately increased to ensure that the model better adapts to the characteristics of these cities, while for cities with relatively similar characteristics, the weight factor may be relatively small so that the model can balance data from more different cities.

[0088] S6: Based on the results of public opinion prediction and analysis, formulate targeted public opinion response strategies for different cities, monitor the development of public opinion in real time, and dynamically adjust the response strategies. Specifically, based on the historical public opinion data and actual public opinion changes of different cities, automatically generate preliminary response strategies. The response strategies include public communication, information release, and policy adjustment to deal with different public opinions. In the process of strategy generation, the geographical location, cultural differences, and socioeconomic characteristics of the city are comprehensively considered. A reinforcement learning algorithm based on policy gradient is used to construct an optimization model for public opinion response strategies. Its objective function is: J(θ) = Eπθ[R(τ)], where π θ is the strategy, R(τ) is the reward value of the public opinion response effect, and the public opinion control effect is optimized by adjusting the model parameters θ.

[0089] The reward value can be defined based on the actual control effect of public opinion, for example, the extent of the decline in public opinion heat, the improvement in public satisfaction, strategy adjustment and parameter optimization: the strategy parameters are updated according to the reward value obtained, with the aim of continuously optimizing the effect of public opinion control.

[0090] First, we predict and analyze public opinion based on historical and real-time data to determine the development trend of public opinion and possible hot spots. Based on the analysis results of public opinion, combined with the historical data, geographical location, cultural differences and socioeconomic characteristics of each city, we generate a preliminary response strategy. This strategy can be a combination of information release, public communication, and policy adjustment. In the process of public opinion development, we continuously adjust the response strategy according to the real-time changes in public opinion. Through the policy gradient method in reinforcement learning, we update the strategy according to the feedback signal of the current public opinion status to ensure the optimal response to public opinion. After each strategy is executed, we evaluate the effectiveness of the response strategy and reward or punish according to the effect. In this way, we gradually optimize the strategy parameters so that we can respond more quickly and effectively when facing similar public opinion events.

[0091] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. The public opinion analysis method based on big data and deep learning is characterized by: include: Collect multi-modal public opinion data including text, images and videos from multiple cities; Perform pre-processing operations such as cleaning, deduplication, word segmentation, and stop word filtering on the collected public opinion data; Use natural language processing technology and computer vision technology to extract text features and image or video features respectively, and fuse these features through a multimodal fusion algorithm; Based on the deep learning framework, we built a multi-city public opinion analysis model, and used historical public opinion data to train the model and optimize the model parameters; Input the public opinion data collected in real time into the trained model to predict the development trend of public opinion in different cities and analyze the hot spots, propagation paths and impact range of public opinion events; Based on the results of public opinion prediction and analysis, targeted public opinion response strategies are formulated for different cities, and the development of public opinion is monitored in real time and the response strategies are adjusted dynamically.

2. The public opinion analysis method based on big data and deep learning as claimed in claim 1, characterized in that: The collecting of multi-modal public opinion data of text, image and video in multiple cities comprises the following steps: Public opinion data collection adopts distributed cross-platform and cross-language data capture, and dynamically adjusts resource allocation through distributed task scheduling. The distributed task scheduling method includes the following steps: Use the public opinion monitoring system to analyze the public opinion heat, discussion volume and dissemination speed indicators of each city in real time, and dynamically adjust resources based on these data to allocate more resources to cities with more active public opinion; A scheduling algorithm based on task priority is adopted to prioritize the public opinion monitoring tasks in cities with high public opinion popularity, ensuring the efficiency of data collection in key cities. The specific formula of the scheduling algorithm is: P(t)=W d ×H(t)+W v ×V(t)+W s ×S(t), P(t) is the priority of city t, H(t) is the public opinion heat of the city, V(t) is the discussion volume of the city, S(t) is the spread speed of the city, W d , W v , W s is the corresponding weight factor, and the optimal configuration of public opinion data collection resources in multiple cities is achieved through the scheduling algorithm.

3. The public opinion analysis method based on big data and deep learning as claimed in claim 1, characterized in that: The pre-processing operations of cleaning, deduplication, word segmentation, and stop word filtering of the collected public opinion data include the following steps: The TF-IDF algorithm is used for keyword extraction, and its calculation formula is as follows: TF(t,d) represents the frequency of term t in document d, DF(t) represents the number of documents containing term t, and N is the total number of documents; For public opinion sentiment analysis, a method based on a combination of sentiment dictionary and deep learning is used, including: Preliminary classification of positive and negative sentiments in the text using a predefined sentiment dictionary; A recurrent neural network model is used to conduct an in-depth analysis of the sentiment tendency of public opinion data. The model input is text data that has been word vectorized, and the output is the public opinion sentiment tendency score.

4. The public opinion analysis method based on big data and deep learning as claimed in claim 3, characterized in that: Use natural language processing technology and computer vision technology to extract text features and image or video features respectively, and fuse these features through a multimodal fusion algorithm. The following steps are involved: Text feature extraction: perform word vectorization on the preprocessed text data, use the word embedding model to obtain word vectors, and use the convolutional neural network to extract high-level text features; Image or video feature extraction: For image data, use pre-trained convolutional neural networks to extract visual features; for video data, extract key frames and use 3D-CNN or LSTM models to obtain spatiotemporal features; Multimodal feature fusion: Fusion of text features with visual features, using feature concatenation, weighted combination or attention mechanism, where the weighted combination formula is: F = αF1 + βF2, where F is the fused feature representation, F1 is the text feature, F2 is the visual feature, α and β are weight coefficients, satisfying α+β=1.

5. The public opinion analysis method based on big data and deep learning as claimed in claim 4, characterized in that: The public opinion analysis model is migrated from the source city to the target city through transfer learning technology. The specific steps include: Source city model training: Train the basic model on the source city with rich data to extract common model parameters, such as word embedding and convolution kernel parameters; Adaptive fine-tuning of target cities: During the migration process, by identifying the differences between the public opinion characteristics of the target city and the source city, the model parameters are dynamically adjusted to improve the adaptability of the model in the new environment; The loss function of transfer learning is: L = λ1L s +λ2L t , where L s , L t are the loss functions of the source city and the target city respectively, and λ1, λ2 are the balancing factors.

6. The public opinion analysis method based on big data and deep learning as claimed in claim 4, characterized in that: The public opinion analysis model also includes a sentiment analysis model, which incorporates the geographical location and cultural difference information of the city into the model. The specific steps are: The geocode of each city is used as one of the inputs of the model to capture the differences between cities; Introducing culturally relevant feature embedding vectors, combined with the city’s historical public opinion data, language habits, and socioeconomic characteristics, to conduct a more fine-grained analysis of public opinion data; When training the model, a multi-task learning framework is used to define multiple loss functions: L i is the loss function of the ith city, λ i is the weight factor for each city, By gradually adjusting the weight of each city, the generalization ability of the model can be improved and the model's adaptability to differences in public opinion between cities can be enhanced.

7. The public opinion analysis method based on big data and deep learning as claimed in claim 1, characterized in that: The method of inputting the real-time collected public opinion data into the trained model to predict the development trend of public opinion in different cities also includes predicting the time dimension of public opinion data based on the long short-term memory network model, specifically including: The time series analysis model is used to predict the time dimension of the public opinion data of each city. The model formula is: h t =f(W h h t-1 +W x x t +b), where h t is the hidden layer state at time t, x t is the input word vector of the current time step, W h , W x is the weight matrix, b is the bias; By combining data on historical events, economic activities and cultural events in each city, a composite prediction model is constructed to predict public opinion hotspots, propagation paths and their scope of influence.

8. The public opinion analysis method based on big data and deep learning as claimed in claim 1, characterized in that: In response to the hot spots, development changes and propagation paths of public opinion in different cities, the system can automatically generate response strategies based on historical data and adjust the response plans in real time through reinforcement learning algorithms, including: Automatically generate preliminary response strategies based on historical public opinion data and actual public opinion changes in different cities. The response strategies include public communication, information release, and policy adjustments to address different public opinions. In the process of strategy generation, the city’s geographical location, cultural differences, and socio-economic characteristics are comprehensively considered; The optimization model of public opinion response strategy is constructed by using the policy gradient-based reinforcement learning algorithm, and its objective function is: J(θ) = Eπθ[R(τ)], where π θ is the strategy, R(τ) is the reward value of the public opinion response effect, and the public opinion control effect is optimized by adjusting the model parameters θ.

9. A method and device for public opinion analysis based on big data and deep learning, characterized in that: The device comprises: Data collection module, used to collect multimodal public opinion data including text, images and videos from multiple cities; The data preprocessing module is used to perform preprocessing operations such as cleaning, deduplication, word segmentation, and stop word filtering on the collected public opinion data; The feature extraction and fusion module is used to extract text features and image or video features respectively using natural language processing technology and computer vision technology, and fuse these features through a multimodal fusion algorithm; The model building module is used to build a multi-city public opinion analysis model based on a deep learning framework, and use historical public opinion data to train the model and optimize model parameters; The public opinion prediction module is used to input the public opinion data collected in real time into the trained model, predict the development trend of public opinion in different cities, and analyze the hot spots, propagation paths, and impact range of public opinion events; The strategy formulation module is used to formulate targeted public opinion response strategies for different cities based on the results of public opinion prediction and analysis, and to monitor the development of public opinion in real time and dynamically adjust the response strategies.

10. A readable storage medium, characterized in that: The readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method according to any one of claims 1 to 8.

Images