Deep learning-based network public opinion evolution simulation method and system

Abstract

The application relates to a deep learning-based network public opinion evolution simulation method and system, and relates to the technical field of text sentiment analysis in natural language processing. The pre-training task of a BERT model is improved, and a deep pre-training task is stacked on the basis, and a LDA topic model is also deeply fused to realize fine-grained public opinion simulation analysis in a topic perspective. A to-be-classified corpus set TC is input into a sentiment classification fine-tuning model to obtain a sentiment classification result, the to-be-classified corpus set TC is input into a BERT model for vectorization processing, the vectorized to-be-classified corpus set TC is input into a LDA topic model for iterative calculation to obtain a document distribution, then the sentiment classification result and the document distribution are fused to obtain a sentiment tendency distribution, the sentiment tendency distribution is divided according to time sequence to obtain a sentiment time sequence simulation result, and therefore more fine-grained and accurate text topic clustering and public opinion evolution simulation results are obtained.

Description

Technical Field

[0001] This invention relates to the field of text sentiment analysis technology in natural language processing, and in particular to a method and system for simulating the evolution of online public opinion during major emergencies based on deep learning. Background Technology

[0002] Traditional methods for simulating the evolution of public opinion sentiment commonly employ sentiment lexicons and supervised machine learning. In recent years, deep learning has gradually replaced traditional sentiment simulation. Long Short-Term Memory (LSTM) neural networks are a special type of recurrent neural network widely used in deep learning. Through embedded forget gates and input gate mechanisms, they alleviate, to some extent, the long-distance dependency problem that general recurrent neural networks (RNNs) cannot solve. However, LSTM remains ineffective for particularly long-term dependencies. Therefore, Vaswani et al. constructed the Transformer model, replacing traditional Word2Vec word vector embedding with a self-attention mechanism for simulation modeling, effectively solving the thorny long-term dependency problem in Natural Language Processing (NLP). Subsequently, Devlin et al. built the BERT pre-trained model based on the Transformer model structure, breaking the highest record on 11 NLP test tasks in the GLUEbenchmark.

[0003] While BERT performs exceptionally well on various NLP tasks, its lack of large-scale sentiment corpora during pre-training presents a bottleneck when performing sentiment analysis. Furthermore, BERT-based sentiment simulation struggles to extract fine-grained thematic insights from the semantics hidden behind massive amounts of text. Summary of the Invention

[0004] The purpose of this invention is to provide a deep learning-based simulation method and system for the evolution of online public opinion. It improves the pre-training task of the BERT model, and on this basis, it superimposes a deep pre-training task. Furthermore, it leverages deep integration with the LDA topic model to achieve fine-grained public opinion simulation analysis from a topic perspective, thereby achieving more accurate public opinion sentiment classification.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A deep learning-based simulation method for the evolution of online public opinion, comprising the following steps:

[0007] Input the training corpus TB into the BERT model;

[0008] The word vectors mapped from the input words in the BERT model are optimized using the LDA topic model to obtain optimized word vectors;

[0009] The optimized word vectors are input into the Transformer encoder of the BERT model to perform sentiment optimization on the pre-training task of the BERT model;

[0010] A single-layer neural network is constructed in the softmax layer of the sentiment-optimized BERT model to serve as a sentiment classifier, thus obtaining the optimized BERT model; the single-layer neural network is connected to the output vector corresponding to the text start character output by the Transformer encoder;

[0011] Using the structure of the optimized BERT model, the training corpus TW is input for deep pre-training to obtain a deep pre-trained model; and the deep pre-trained model is then transferred to the fine-tuning model in the softmax layer of the optimized BERT model to obtain a sentiment classification fine-tuning model.

[0012] Input the corpus TC to be classified into the sentiment classification fine-tuning model to obtain the sentiment classification result;

[0013] The corpus set TC to be classified is input into the BERT model for vectorization processing, and the vectorized corpus set TC is input into the LDA topic model for iterative calculation to obtain the document distribution.

[0014] By combining the sentiment classification results and the document distribution, a sentiment tendency distribution is obtained;

[0015] The distribution of emotional tendencies is divided according to time sequence to obtain the emotional time sequence simulation results.

[0016] The present invention also provides a deep learning-based simulation system for the evolution of online public opinion, the simulation system comprising:

[0017] The input module is used to input the training corpus TB into the BERT model;

[0018] The word vector optimization module is used to optimize the word vectors mapped to the input words in the BERT model using the LDA topic model, so as to obtain the optimized word vectors.

[0019] The sentiment optimization module is used to input the optimized word vectors into the Transformer encoder of the BERT model to perform sentiment optimization on the pre-training task of the BERT model.

[0020] The BERT model acquisition module is optimized to construct a single-layer neural network in the softmax layer of the sentiment-optimized BERT model as a sentiment classifier to obtain the optimized BERT model; the single-layer neural network is connected to the output vector corresponding to the text start character output by the Transformer encoder;

[0021] The sentiment classification fine-tuning model acquisition module is used to use the structure of the optimized BERT model, input the training corpus TW for deep pre-training, and obtain a deep pre-trained model; and then transfer the deep pre-trained model to the fine-tuning model in the softmax layer of the optimized BERT model to obtain the sentiment classification fine-tuning model.

[0022] The sentiment classification result acquisition module is used to input the corpus TC to be classified into the sentiment classification fine-tuning model to obtain the sentiment classification result;

[0023] The document distribution acquisition module is used to input the corpus set TC to be classified into the BERT model for vectorization processing, and then input the vectorized corpus set TC to be classified into the LDA topic model for iterative calculation to obtain the document distribution.

[0024] The sentiment distribution acquisition module is used to fuse the sentiment classification results and the document distribution to obtain the sentiment distribution.

[0025] The emotional time-series simulation result acquisition module is used to divide the emotional tendency distribution according to time sequence to obtain the emotional time-series simulation results.

[0026] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0027] This invention provides a deep learning-based method and system for simulating the evolution of online public opinion. The training corpus (TB) is input into a BERT model, and the word vectors mapped from the input words in the BERT model are optimized using the LDA topic model. This optimized word vector not only fully incorporates contextual semantic information to compensate for the shortcomings of the LDA bag-of-words model, training better topic vectors, but also deeply integrates the optimized topic vectors with the word vectors of the BERT model, achieving fine-grained public opinion simulation analysis from a topic perspective. The optimized word vectors are then input into the Transformer encoder of the BERT model to perform sentiment optimization on the pre-training task, enabling the model to learn more sentiment information. A single-layer neural network is constructed in the softmax layer of the sentiment-optimized BERT model as a sentiment classifier, resulting in the optimized BERT model. By utilizing the optimized BERT model structure, the training corpus TW is input for deep pre-training to obtain a deep pre-trained model. This deep pre-trained model is then transferred to the fine-tuning model in the softmax layer of the optimized BERT model to obtain a sentiment classification fine-tuning model. This allows for the superposition of deep pre-training tasks on top of the improved BERT model pre-training task, achieving more accurate public opinion sentiment classification.

[0028] The corpus TC to be classified is input into the sentiment classification fine-tuning model to obtain the sentiment classification result. Then, the corpus TC to be classified is input into the BERT model for vectorization processing. The vectorized corpus TC to be classified is then input into the LDA topic model for iterative calculation to obtain the document distribution. Then, the sentiment classification result and the document distribution are fused to obtain the sentiment tendency distribution. The sentiment tendency distribution is then divided according to the time sequence to obtain the sentiment time series simulation result, thus obtaining more fine-grained and accurate text topic clustering and public opinion evolution simulation results. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of the simulation method provided in Embodiment 1 of the present invention.

[0031] Figure 2 This is a schematic diagram of the structure of the improved BERT model provided in Embodiment 1 of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] The purpose of this invention is to provide a deep learning-based simulation method and system for the evolution of online public opinion. It improves the pre-training task of the BERT model, and on this basis, it superimposes a deep pre-training task. Furthermore, it leverages deep integration with the LDA topic model to achieve fine-grained public opinion simulation analysis from a topic perspective, thereby achieving more accurate public opinion sentiment classification.

[0034] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] Example 1:

[0036] This embodiment provides a deep learning-based simulation method for the evolution of online public opinion, such as... Figure 1 As shown, the simulation method includes the following steps:

[0037] Step 101: Input the training corpus TB into the BERT model;

[0038] Before inputting the training corpus TB into the BERT model, the simulation method further includes data preprocessing of the training corpus TB, including word segmentation and stop word removal. It can also statistically analyze the word frequency information of each word in the training corpus TB, extract high-frequency words, and add special characters to form a complex corpus dictionary, with each word corresponding to a unique index.

[0039] Step 102: Optimize the word vectors mapped to the input words in the BERT model using the LDA topic model to obtain optimized word vectors;

[0040] BERT (Bidirectional Encoder Representations from Transformers) is a language pre-training model released by Google in 2018 based on a bidirectional Transformer structure. It enhances the model's semantic representation capabilities through Masked Language Modeling (MLM) and Next Sentence Prediction (NSP) tasks, and leverages the powerful feature extraction and fine-tuning transfer learning capabilities of the Transformer to excel in various NLP tasks. To improve the accuracy and fine-grainedness of large-scale complex text in sentiment classification tasks, this embodiment retains the basic BERT model while embedding the topic optimization results of the LDA topic model into the pre-training and fine-tuning stages of the BERT model. This allows the BERT model to simultaneously learn syntactic, semantic, and topic-related text features when performing sentiment classification tasks. The structure of the improved BERT model is as follows: Figure 2 As shown in Table 1, ω, δ, and ρ are defined as the word embedding, segment embedding, and position embedding of the text obtained by the BERT model, respectively. Trm represents the Transformer encoder. For special characters inserted into the text, this embodiment provides the parsing as shown in Table 1.

[0041] Table 1. Mapping Relationships of Special Characters in Text Analysis

[0042]

[0043] like Figure 2 As shown, after inputting the segmented document into the model, each word is mapped to three vectors, representing w. ij (ω+δ+ρ), collectively referred to as word vectors. During the improvement process, to enable the BERT model to achieve fine-grained public opinion simulation analysis from a topic perspective, the word vectors w... ijThe (ω+δ+ρ) method is combined with the topic representation of the LDA topic model. However, LDA (Latent Dirichlet Allocation) is a commonly used document topic extraction optimization model. It provides the topic of each document in the text set in the form of a probability distribution, and then optimizes topic clustering or text classification based on the topic distribution. Subsequently, the GibbsSampling algorithm is used for parameter estimation, and iterative sampling is performed until convergence to obtain the "topic-word" distribution matrix and the "text-topic" distribution. In the calculation process of the original model, the topic vector μ is calculated from the cosine distance between the high-frequency words of each topic and the document. The topic words are limited by the traditional structure of the LDA bag-of-words model, which cannot effectively combine the semantic and contextual information of the text, and the quality of word segmentation technology has a significant impact on the calculation result of μ. In order to obtain a better topic vector, this embodiment integrates the word vectors in the BERT model with the topic representation of the LDA model, and obtains the optimized topic vector u' by iteratively calculating the word weights, in order to provide more accurate topic semantic information for sentiment simulation of large-scale complex texts.

[0044] Specifically, the optimization of word vectors mapped from input words in the BERT model using the LDA topic model to obtain optimized word vectors includes: inputting the training corpus TB into the BERT model and performing vectorization processing, where each word in the training corpus TB is mapped into a word vector containing a sense vector, a text vector, and a position vector; inputting the vectorized training corpus TB into the LDA topic model to obtain optimized topic vectors; embedding the optimized topic vectors into the BERT model to optimize the word vectors mapped from input words in the BERT model, resulting in optimized word vectors; the optimized word vectors include sense vectors, text vectors, position vectors, and topic vectors. This leads to a word vector w that incorporates better topic vectors. ij (ω+δ+ρ+u'), and then pass it into the bidirectional Transformer encoder.

[0045] Step 103: Input the optimized word vectors into the Transformer encoder of the BERT model to perform sentiment optimization on the pre-training task of the BERT model;

[0046] The construction of the public opinion sentiment evolution simulation model aims to accurately simulate the sentiment evolution process of large-scale online public opinion, grasp the characteristics and patterns of group sentiment evolution, and assist government departments in quickly forming an effective public opinion response mechanism. Because BERT lacks training with sentiment corpora during the pre-training phase, its performance in sentiment classification tasks is poor. Therefore, this embodiment proposes a new pre-training task for the BERT model, introducing an improved pre-training corpus TB. The training corpus TB includes Chinese Wikipedia corpus as well as sentiment corpora from Sina Weibo and Baidu Tieba. Specifically, it adds Sina Weibo and Baidu Tieba sentiment corpora to the original Chinese Wikipedia corpus, hoping that the model can learn more sentiment information and better perform sentiment classification tasks.

[0047] Furthermore, to enable the model to learn more information, the Transformer encoder connects the multi-head attention mechanism and the feed-forward layer through a residual network structure. The multi-head attention mechanism performs multiple linear transformations on the input vector to obtain different linear values, and then calculates the attention weights. The calculation process is shown in equations (1) and (2):

[0048] MultiHead(Q,K,V)=Concat(head1, head2,..., head h W O (1)

[0049]

[0050] In the formula, Q, K, and V are the input word vector matrices. The Attention is calculated after mapping Q, K, and V through a parameter matrix. This process is repeated h times, and the results are then concatenated. f ={head f |f∈{1,2,...,h}} represents the f-th hyperparameter header, W O This is the weight matrix. This represents W corresponding to the f-th hyperparameter header. Q W K W V Weight matrix.

[0051] Thus, the Transformer encoder learns and stores document d. i The semantic relationships and syntactic structure information, due to document d i After the BERT model was improved and incorporated with better topic vectors, the improved document d′ was used. iThe model utilizes a special character [CLS] connected to the softmax output layer to adapt to transfer learning in multi-task scenarios. This combination of feature vectors incorporating topic optimization information and BERT word vectors is expected to improve the model's accuracy and fine-grainedness in large-scale, complex text sentiment evolution simulations.

[0052] Step 104: Construct a single-layer neural network in the softmax layer of the sentiment-optimized BERT model as a sentiment classifier to obtain the optimized BERT model; the single-layer neural network is connected to the output vector corresponding to the text start character [CLS] output by the Transformer encoder;

[0053] The sentiment classifier is used to perform the sentiment classification (SC) task, while the sentiment-optimized BERT model still retains the MLM and NSP tasks. The softmax layer of the BERT model also includes a single-layer neural network connected to the output vector corresponding to the text start character output by the Transformer encoder to perform the NSP task. The softmax layer of the BERT model also includes a single-layer neural network connected to the output vector corresponding to the masked character output by the Transformer encoder to perform the MLM task.

[0054] Step 105: Using the structure of the optimized BERT model, input the training corpus TW for deep pre-training to obtain a deep pre-trained model; and transfer the deep pre-trained model to the fine-tuning model in the softmax layer of the optimized BERT model to obtain the sentiment classification fine-tuning model.

[0055] This embodiment improves upon the existing pre-training task by adding a deep pre-training task to achieve more accurate public opinion sentiment classification. It introduces publicly available Sina Weibo annotation sets and a small number of event-specific sentiment annotation sets (TW) as BERT deep pre-training corpora.

[0056] Specifically, the process of using the optimized BERT model structure and inputting the training corpus TW for deep pre-training to obtain a deep pre-trained model includes:

[0057] Set the initial learning rate (LR), batch size, dropout rate, and number of epochs.

[0058] The sum of the losses of the MLM task, the NSP task, and the SC task is used as the loss function for deep pre-training.

[0059] The loss function is:

[0060] Loss = λ1LossMLM +λ2Loss NSP +λ3Loss SC (3)

[0061] In Equation 3, λ1, λ2, and λ3 represent the weight assignments for the MLM task, NSP task, and SC task, respectively.

[0062] The training corpus TW is used for deep pre-training. During the training process, the Adam optimization algorithm is used to determine the optimal learning rate LR, AUC is used as the indicator to evaluate the training effect of the model, and F1 score is used to determine the optimal threshold for positive and negative sentiment classification in order to avoid overfitting during training, thus obtaining a deep pre-trained model.

[0063] Step 106: Input the corpus TC to be classified into the sentiment classification fine-tuning model to obtain the sentiment classification result;

[0064] Here, TC represents the corpus for a sentiment classification task specific to a particular event. The input is the corpus TC to be classified, and TC is calculated for each document. i Emotional value E(TC) i Based on the sentiment classification threshold, the sentiment value is converted into the corresponding sentiment polarity E'(TC). i This yields the coarse-grained sentiment classification result RTC; RTC = (MnE) pos (TC i MnE neg (TC i ), E' pos (TC i ), E' neg (TC i )); where MnE pos (TC i MnE represents the mean of positive sentiment. neg (TC i E' represents the mean of negative sentiment. pos (TC i E' represents the number of texts with positive sentiment. neg (TC i () represents the number of negative sentiment texts.

[0065] Then, the RTCs are integrated into the time series text set TC_time in chronological order, where TC_time = {(TC1, time1), (TC2, time2), ..., (TC...}. M time q)}; where time contains q time slices, yielding a coarse-grained "text_time" sentiment time-series evolution result time_RTC; time_RTC={time_RTC1, time_RTC2, ..., time_RTC q}

[0066] Step 107: Input the corpus set TC to be classified into the BERT model for vectorization, and then input the vectorized corpus set TC into the LDA topic model for iterative calculation to obtain the document distribution;

[0067] The corpus TC to be classified is input into the BERT model for vectorization processing. Each word in the corpus TC is mapped to a word vector containing a semantic vector, a text vector, and a position vector. The vectorized corpus TC is then input into the LDA topic model to obtain the document distribution TTC corresponding to the "topic-word" at l optimal topic numbers, where TTC = {(TC1, l1), (TC2, l2), ..., (TC1, l1)}. M , l l )}.

[0068] Step 108: Combine the sentiment classification results and the document distribution to obtain the sentiment tendency distribution;

[0069] By combining the document distribution TTC with the coarse-grained sentiment classification result RTC, we obtain the sentiment tendency distribution TTC' of "topic-word".

[0070] Step 109: Divide the emotional tendency distribution according to time sequence to obtain the emotional time sequence simulation results.

[0071] By integrating the sentiment distribution TTC' into the time-series text set TC_time, a fine-grained "text-topic" sentiment time-series simulation result time_TTC' is obtained; time_TTC' = {time_TTC'1, time_TTC'2, ..., time_TTC'} q}

[0072] The online public opinion sentiment simulation method provided in this embodiment integrates the LDA topic model and the BERT model. It can not only fully combine contextual semantic information and make up for the shortcomings of the LDA bag-of-words model to train better topic vectors, but also deeply integrate the optimized topic vectors with BERT word vectors to obtain more fine-grained and accurate text topic clustering and public opinion evolution simulation results.

[0073] Example 2:

[0074] This embodiment provides a deep learning-based network public opinion evolution simulation system, which operates using the simulation method described in Embodiment 1. The simulation system includes:

[0075] The input module is used to input the training corpus TB into the BERT model;

[0076] The word vector optimization module is used to optimize the word vectors mapped to the input words in the BERT model using the LDA topic model, so as to obtain the optimized word vectors.

[0077] The sentiment optimization module is used to input the optimized word vectors into the Transformer encoder of the BERT model to perform sentiment optimization on the pre-training task of the BERT model.

[0078] The BERT model acquisition module is optimized to construct a single-layer neural network in the softmax layer of the sentiment-optimized BERT model as a sentiment classifier to obtain the optimized BERT model; the single-layer neural network is connected to the output vector corresponding to the text start character output by the Transformer encoder;

[0079] The sentiment classification fine-tuning model acquisition module is used to use the structure of the optimized BERT model, input the training corpus TW for deep pre-training, and obtain a deep pre-trained model; and then transfer the deep pre-trained model to the fine-tuning model in the softmax layer of the optimized BERT model to obtain the sentiment classification fine-tuning model.

[0080] The sentiment classification result acquisition module is used to input the corpus TC to be classified into the sentiment classification fine-tuning model to obtain the sentiment classification result;

[0081] The document distribution acquisition module is used to input the corpus set TC to be classified into the BERT model for vectorization processing, and then input the vectorized corpus set TC to be classified into the LDA topic model for iterative calculation to obtain the document distribution.

[0082] The sentiment distribution acquisition module is used to fuse the sentiment classification results and the document distribution to obtain the sentiment distribution.

[0083] The emotional time-series simulation result acquisition module is used to divide the emotional tendency distribution according to time sequence to obtain the emotional time-series simulation results.

[0084] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0085] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A deep learning-based simulation method for the evolution of online public opinion, characterized in that, The simulation method includes the following steps: Input the training corpus TB into the BERT model; The word vectors mapped from the input words in the BERT model are optimized using the LDA topic model to obtain optimized word vectors; The optimized word vectors are input into the Transformer encoder of the BERT model to perform sentiment optimization on the pre-training task of the BERT model; A single-layer neural network is constructed in the softmax layer of the sentiment-optimized BERT model to serve as a sentiment classifier, resulting in the optimized BERT model; the single-layer neural network is connected to the output vector corresponding to the text start character output by the Transformer encoder; Using the structure of the optimized BERT model, the training corpus TW is input for deep pre-training to obtain a deep pre-trained model; and the deep pre-trained model is then transferred to the fine-tuning model in the softmax layer of the optimized BERT model to obtain a sentiment classification fine-tuning model. Input the corpus TC to be classified into the sentiment classification fine-tuning model to obtain the sentiment classification result; The corpus set TC to be classified is input into the BERT model for vectorization processing, and the vectorized corpus set TC is input into the LDA topic model for iterative calculation to obtain the document distribution. By combining the sentiment classification results and the document distribution, a sentiment tendency distribution is obtained; The distribution of emotional tendencies is divided according to time sequence to obtain the emotional time sequence simulation results.

2. The network public opinion evolution simulation method based on deep learning as described in claim 1, characterized in that, The training corpus TB includes Chinese Wikipedia corpus as well as sentiment corpus from Sina Weibo and Baidu Tieba.

3. The deep learning-based network public opinion evolution simulation method as described in claim 1, characterized in that, Before inputting the training corpus TB into the BERT model, the simulation method also includes data preprocessing of the training corpus TB, including word segmentation and stop word removal.

4. The network public opinion evolution simulation method based on deep learning as described in claim 1, characterized in that, The optimization of word vectors mapped from input words in the BERT model using the LDA topic model, resulting in optimized word vectors, specifically includes: The training corpus TB is input into the BERT model and vectorized. Each word in the training corpus TB is mapped into a word vector containing a word meaning vector, a text vector, and a position vector. The vectorized training corpus TB is input into the LDA topic model to obtain optimized topic vectors. The optimized topic vectors are then embedded into the BERT model. The word vectors mapped to the input words in the BERT model are optimized to obtain optimized word vectors. The optimized word vectors include semantic vectors, text vectors, position vectors, and topic vectors.

5. The network public opinion evolution simulation method based on deep learning as described in claim 1, characterized in that, The sentiment classifier is used to perform the SC task.

6. The network public opinion evolution simulation method based on deep learning as described in claim 5, characterized in that, The softmax layer of the BERT model also includes a single-layer neural network connected to the output vector corresponding to the text start character output by the Transformer encoder, in order to perform the NSP task.

7. The network public opinion evolution simulation method based on deep learning as described in claim 6, characterized in that, The softmax layer of the BERT model also includes a single-layer neural network connected to the output vector corresponding to the masking word output by the Transformer encoder, in order to perform the MLM task.

8. The network public opinion evolution simulation method based on deep learning as described in claim 7, characterized in that, The process of using the optimized BERT model structure and inputting the training corpus TW for deep pre-training to obtain the deep pre-trained model specifically includes: Set the initial learning rate, batch size, dropout rate, and number of epochs; The sum of the losses of the MLM task, the NSP task, and the SC task is used as the loss function for deep pre-training. The training corpus TW is used for deep pre-training. During the training process, the Adam optimization algorithm is used to determine the optimal learning rate, AUC is used as the indicator to evaluate the training effect of the model, and F1 score is used to determine the optimal threshold for positive and negative sentiment classification, thus obtaining the deep pre-trained model.

9. The network public opinion evolution simulation method based on deep learning as described in claim 1, characterized in that, The Transformer encoder connects the multi-head mechanism and the feedforward layer through a residual network structure.

10. A deep learning-based simulation system for the evolution of online public opinion, characterized in that, The simulation system includes: The input module is used to input the training corpus TB into the BERT model; The word vector optimization module is used to optimize the word vectors mapped to the input words in the BERT model using the LDA topic model, so as to obtain the optimized word vectors. The sentiment optimization module is used to input the optimized word vectors into the Transformer encoder of the BERT model to perform sentiment optimization on the pre-training task of the BERT model. The BERT model acquisition module is optimized to construct a single-layer neural network in the softmax layer of the sentiment-optimized BERT model as a sentiment classifier to obtain the optimized BERT model; the single-layer neural network is connected to the output vector corresponding to the text start character output by the Transformer encoder; The sentiment classification fine-tuning model acquisition module is used to use the structure of the optimized BERT model, input the training corpus TW for deep pre-training, and obtain a deep pre-trained model; and then transfer the deep pre-trained model to the fine-tuning model in the softmax layer of the optimized BERT model to obtain the sentiment classification fine-tuning model. The sentiment classification result acquisition module is used to input the corpus TC to be classified into the sentiment classification fine-tuning model to obtain the sentiment classification result; The document distribution acquisition module is used to input the corpus set TC to be classified into the BERT model for vectorization processing, and then input the vectorized corpus set TC to be classified into the LDA topic model for iterative calculation to obtain the document distribution. The sentiment distribution acquisition module is used to fuse the sentiment classification results and the document distribution to obtain the sentiment distribution. The emotional time-series simulation result acquisition module is used to divide the emotional tendency distribution according to time sequence to obtain the emotional time-series simulation results.

Images