An opinion guidance method and device based on an opinion sentiment text automatic generation model

Abstract

The application discloses a public opinion guiding method and device based on a public opinion sentiment text automatic generation model, relates to the field of information technology processing, and comprises the following steps: firstly, a public opinion sentiment text automatic generation model is trained, the public opinion sentiment text automatic generation model further comprises the following steps: public opinion text data is preprocessed by removing noise; a sentiment analysis module is trained; a theme extraction module is trained; a public opinion text generation module is trained; and then, positive public opinion text is generated by using the public opinion sentiment text automatic generation model and is automatically input into a network platform, so as to divert current contradictions and guide public opinion attention. The method can automatically generate a large amount of positive public opinion sentiment text with a specified theme and a specified sentiment, dilutes and relieves negative public opinion and emotions, and has a good public opinion guiding effect.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, specifically to a method and apparatus for guiding public opinion based on an automatic generation model of public opinion sentiment text. Background Technology

[0002] Social network public opinion guidance refers to the process of researching and analyzing online public opinion concerning a specific event, topic, or individual using various online media methods, and then guiding, shaping, or controlling the development trend of public opinion through a series of strategies and means to achieve specific goals or effects. This process can include stages such as public opinion monitoring, public opinion analysis, and public opinion shaping. In modern society, guiding online public opinion has become one of the important tasks of various institutions and organizations.

[0003] To cultivate a positive online public opinion environment and guide public opinion in the right or desired direction, a large number of texts with positive and constructive effects can be released in a short period. Alternatively, texts that guide the public and shift attention can be published. Various methods can be used to change people's perception of negative and malicious public opinion, thereby further reversing the course of public opinion. Therefore, researching and designing methods for automatically generating online commentary texts based on intelligent writing technology has significant research and application value.

[0004] There are many methods for guiding public opinion, the most widespread being text generation models that generate text to counter malicious information on social networks. This combines topic models and deep learning models to generate text that reflects the desired public opinion direction. However, the generated text lacks emotional attributes, and text with emotional bias can better guide online public opinion. To address this issue, the paper "Research on Text Generation Based on GAN Considering External Knowledge and Emotional Attributes" (author Ruan Yeli, Zhongnan University of Economics and Law) proposed integrating emotional attributes into the generated text, achieving better results. However, current methods for generating text for guiding online public opinion based on collaborative modeling of public opinion topics and emotional attributes are limited. Furthermore, existing text generation models cannot simultaneously address the three aspects of specifying a topic, specifying an emotion, and automatic generation. Based on these shortcomings, this invention proposes a method and device for guiding public opinion based on an automatic generation model of emotional text. Summary of the Invention

[0005] This invention aims to solve at least one of the technical problems existing in the prior art. To this end, this invention proposes a method and apparatus for guiding public opinion based on an automatic generation model of public opinion sentiment text.

[0006] To achieve the above objectives, this invention proposes a public opinion guidance method based on an automatic generation model of public opinion sentiment text, comprising: The training process for an automatic public opinion sentiment text generation model involves: preprocessing the public opinion text data by denoising it; training the sentiment analysis module, which uses a sentiment classifier to categorize the denoised public opinion text into three sentiment classes: positive, neutral, and negative, and public opinion texts in these categories; training the topic extraction module, which uses a topic extraction algorithm to extract topics from the positive, neutral, and negative sentiment texts, resulting in positive sentiment topic P and negative sentiment topic N; and training the public opinion text generation module, which inputs the topics and corresponding sentiment texts into the text generation model to generate public opinion texts with positive, neutral, and negative sentiments. Generate positive public opinion text: Input negative public opinion topics into the positive public opinion text generator trained by the public opinion sentiment text automatic generation model, and a large number of positive public opinion texts corresponding to negative public opinion topics can be quickly obtained for the network public opinion guidance module to call. The module for guiding online public opinion automatically feeds positive public opinion texts into online platforms to divert attention from current conflicts and guide public opinion.

[0007] Furthermore, the RoBERTa model is used to train a sentiment classifier, including: using a deep learning model to annotate hundreds of thousands of public opinion text data to obtain an annotated dataset; applying a softmax neural network model to train a positive, neutral, and negative sentiment classifier using the annotated sentiment data; and classifying the social network public opinion texts into positive, neutral, and negative texts respectively.

[0008] Furthermore, the TextRank algorithm is used to calculate positive and negative topic sentiment texts separately, extracting positive and negative topic words. The algorithm is as follows: in, Representative sentence The weights are calculated, and the summation on the right represents the contribution of each adjacent sentence to the current sentence. In a single document, all sentences can be roughly considered to be adjacent, and there is no need to generate and extract multiple windows as in multiple documents; only a single document window is needed. Indicates the similarity between two sentences. Represents the sentence generated in the last iteration. The weights; It is the damping coefficient, which is typically 0.85.

[0009] Furthermore, the ERNIE-GEN model is used to generate text. The training of the ERNIE-GEN model includes: supervised training on a large corpus of data used for pre-training the RoBERTa model; constructing pre-training data by combining multi-granularity target sampling to enhance the correlation between the encoder and decoder; compensating for the difference between training and inference through a padding generation mechanism and a noise-aware generation method; and using a multi-stream attention mechanism, adding a span-by-span generation stream to the word-by-word generation stream, allowing the model to generate a semantically complete segment each time. The formula for the word-by-word generation stream is as follows: Where Q, K, and V represent the query, key, and value in MultiHeadAttention, MH-Attn. It is the first Noisy target sequences in layers, Refers to the first Layer source sequence, Indicates the first The first layer There are two vector representations, where ai represents the i-th vector representation of the artificial symbol sequence AW in the l-th layer. The formula for span-by-span generation flow is as follows: in, Represents the i-th span, where , Indicates the first A span Vector representation.

[0010] Furthermore, the training of the public opinion text production model includes: using positive sentiment keywords extracted by the Textrank algorithm as labels, and positive sentiment blog posts as text input to the ERNIE-GEN model for training; training ERNIE-GEN using a positive public opinion generation dataset to obtain a positive public opinion text generator; modeling public opinion topics after sentiment classification and topic extraction; and inputting negative public opinion topics into the positive text generator to obtain generated positive text.

[0011] This invention also proposes a network public opinion guidance device based on an automatic generation model of public opinion sentiment text, comprising: The ERNIE-GEN model training module is used to denoise the topic information and comment information of public opinion on online social platforms, use the RoBERTa sentiment classifier to divide the public opinion text into different sentiment categories, use the TextRank algorithm to calculate the sentiment text, extract the sentiment topic words, and input them into the specified model training module to obtain different sentiment public opinion text generators, which can be called by the positive public opinion text generation module. The positive public opinion text generation module uses the ERNIE-GEN model to generate a specified number of positive public opinion texts, and then stores and sends these texts to the positive public opinion text delivery module for the module to use. The positive public opinion text delivery module is used to call or simulate the public opinion release function of the corresponding public opinion platform based on the identity key of each important public opinion platform filled in by the user, and release the generated positive public opinion text, thereby guiding negative public opinion and diverting public attention.

[0012] Compared with the prior art, the beneficial effects of the present invention are: 1. Compared with traditional extraction-based generation methods, the proposed method can automatically generate the required text; 2. Compared with other automatic text generation methods, it can generate text with a positive and proactive emotional orientation based on the public opinion theme and the desired emotion to guide online public opinion, divert public attention and focus, and thus achieve the effect of guiding public opinion. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of a public opinion guidance method based on an automatic generation model of public opinion sentiment text according to the present invention; Figure 2 A schematic diagram of the structure of the automatic generation model for public opinion sentiment text for implementation cases; Figure 3 ERNIE-GEN model framework diagram for implementation case; Figure 4 A comparison chart of the evaluation results of the sentiment classification comparison model in the example; Figure 5 The text generation comparison model evaluation results are shown in the example chart. Detailed Implementation

[0015] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] According to embodiments of the present invention, a method and apparatus for guiding public opinion based on an automatic generation model of public opinion sentiment text is proposed, such as... Figure 1 As shown, the process includes: training an automatic public opinion sentiment text generation model, specifically, performing noise reduction preprocessing on public opinion text data; training a sentiment analysis module, using a sentiment classifier to divide the noise-reduced public opinion text into sentiment categories, obtaining three sentiment classifiers (positive, neutral, and negative) and public opinion texts with positive, neutral, and negative sentiment categories; training a topic extraction module, using a topic extraction algorithm to extract topics from the positive, neutral, and negative sentiment texts, obtaining positive sentiment topic P and negative sentiment topic N; and training a public opinion text generation module, inputting the topics and corresponding sentiment texts into the text generation model for training, obtaining three sentiment public opinion text generators (positive, neutral, and negative). Generate positive public opinion text: Input negative public opinion topics into the positive public opinion text generator trained by the public opinion sentiment text automatic generation model, and a large number of positive public opinion texts corresponding to negative public opinion topics can be quickly obtained for the network public opinion guidance module to call. The module for guiding online public opinion automatically feeds positive public opinion texts into online platforms to divert attention from current conflicts and guide public opinion.

[0017] Among them, such as Figure 2 As shown, taking a certain public opinion event as an experimental case, the public opinion sentiment text automatic generation model includes three modules: (1) Sentiment analysis module: using the RoBERTa sentiment classifier to identify the sentiment polarity of the original Weibo posts of the public opinion event, and obtaining blog posts with positive, neutral and negative sentiments; (2) Topic extraction module: using the TextRank algorithm to extract the topics of the blog posts with positive, neutral and negative sentiments, and obtaining positive sentiment topic P and negative sentiment topic N; (3) Public opinion text generation module: positive public opinion topics and blog posts are input into ERNIE-GEN for training the text generator, and negative public opinion topics are input into the trained positive sentiment text generator to obtain the positive sentiment blog posts TN corresponding to the negative sentiment topic N.

[0018] The core of RoBERTa is the Multi-HeadAttention mechanism in the Transformer structure. The Attention mechanism can be understood as follows: given a set of vectors Value (V), Key (K), and Query (Q), the similarity between Query and Key is calculated, and the vectors in Key that are most relevant to Query are given higher weights. This achieves focused processing of information in Key. Finally, the weights and Value are summed in a weighted manner.

[0019] The TextRank algorithm treats linguistic units of text (such as words or sentences) as nodes in a graph, and the relationships between them as edges. Then, using iterative computation, it identifies the most relevant nodes in the text by continuously updating the weight value of each node. These nodes can help generate text summaries or keyword lists. The algorithm is as follows: Here, represents the sentence weight, and the sum on the right represents the contribution of each adjacent sentence to this sentence. In a single document, all sentences can be roughly considered to be adjacent, eliminating the need for multiple windows to be generated and extracted as in multiple documents; a single document window is sufficient. represents the similarity between two sentences, represents the weight of the sentence from the previous iteration, and is a damping coefficient, typically 0.85.

[0020] like Figure 3 As shown, the text generation stage uses the ERNIE-GEN model. The original ERNIE-GEN used a pre-trained large language model (BERT) as a fine-tuning framework, which this invention replaces with RoBERTa. The pre-trained large language model has already been trained using a large amount of corpus data, and ERNIE-GEN continues to undergo supervised training to further perform specific text generation tasks.

[0021] ERNIE-GEN combines multi-granularity target sampling to construct pre-training data, enhancing the correlation between the encoder and decoder. This framework bridges the gap between training and inference through padding generation mechanisms and noise-aware generation methods. Furthermore, it employs a multi-stream attention mechanism during training. Building upon word-by-word generation, it adds span-by-span generation, enabling the model to generate a semantically complete segment at a time.

[0022] Word-by-word generation formula: Where Q, K, and V represent the query, key, and value in MultiHeadAttention (MH-Attn), is the noisy target sequence of the layer, refers to the source sequence of the layer, represents the vector representation of the layer; ai represents the ith vector representation of the artificial symbol sequence AW of the l-th layer.

[0023] Span-by-span generation flow formula: in, Represents the i-th span, where , Indicates the first A span Vector representation.

[0024] This invention trains the ERNIE-GEN text generator within its proposed framework, using positive sentiment keywords extracted from TextRank as labels and positive sentiment blog posts as text input to the ERNIE-GEN model for training. The ERNIE-GEN model is trained using a positive public opinion generation dataset to obtain a positive public opinion text generator. After sentiment classification and topic extraction, public opinion topics are modeled. Specifically, the RoBERTa sentiment classifier categorizes public opinion texts into negative, neutral, and positive sentiment texts. Then, the topic extraction module uses the TextRank algorithm to extract topics from the sentiment texts, obtaining the topic for each sentiment text. The data concatenated with the positive topics and texts is input into the ERNIE-GEN model to obtain the desired positive text generator, P-GEN. Based on the desired negative public opinion topics, the positive P-GEN text generator is input to generate positive text. This positive sentiment text is then released into social networks to purify the online environment, amplifying the influence of positive public opinion through the generated public opinion, changing the online public opinion landscape, and even reversing the situation.

[0025] This invention utilizes the dataset from the 26th National Conference on Information Retrieval (CCIR2020) evaluation competition to construct the required sentiment text generation dataset. This dataset was collected by the CCIR2020 organizers based on 230 relevant keywords, capturing a total of 1 million Weibo posts from January 1st to February 19th, 2020. Sentiment annotations were then applied to 100,000 of these posts, categorized into positive, neutral, and negative sentiment. This resulted in 100,000 annotated sentiment data points and 900,000 unannotated sentiment data points. For the sentiment analysis module, this invention uses RoBERTa to train the required sentiment classifier for epidemic-related public opinion texts. First, the RoBERTa model is fine-tuned using the 100,000 annotated sentiment data points. The 100,000 annotated data points are then divided into training, validation, and test sets in a 6:2:2 ratio. The RoBERTa model is used to train the training set, validated using the validation set, and tested using the test set.

[0026] The data on Weibo contains a lot of noise. Text generation models have high requirements for data quality. Therefore, data preprocessing can help provide better performance for pre-trained generation models. To prevent excessive noise and irrelevant content in the text generation data, this invention performs the following data preprocessing steps based on the HarvestText toolkit: (1) Remove @+username, emoticons, and email address "abc@demo.com" from the text; (2) Remove unnecessary spaces and line breaks; (3) Remove escaped HTML tags; (4) Replace hyperlinks mentioned in the text with URLs; (5) Convert traditional Chinese characters in the dataset to simplified Chinese characters.

[0027] This invention conducted experiments with seven sets of comparative models: ELECTRA + ERNIE-GEN, ELECTRA-small + ERNIE-GEN, ERNIE-1.0 + ERNIE-GEN, ERNIE-tiny + ERNIE-GEN, BERT + ERNIE-GEN, MacBERT + ERNIE-GEN, and RoBERTa + ERNIE-GEN. First, a pre-trained language model was used to perform sentiment classification on Weibo sentiment texts. Then, the generation task was trained and computed based on this model. Accuracy is obtained by performing sentiment classification on the dataset using the pre-trained language model, and the Rouge metric is obtained from the generation task using both the pre-trained language model and ERNIE-GEN.

[0028] like Figure 4As shown in the results of the sentiment classification experiment, the ELECTRA-small and ERNIE-tiny models had the lowest sentiment classification accuracy (72.32% and 72.91%, respectively), while the RoBERTa model had the highest sentiment classification accuracy (76.08%). This is because RoBERTa was pre-trained with more data and has more parameters, reflecting that smaller-level language models are not as good as larger-level language models in classification tasks. Therefore, RoBERTa performs best as the sentiment classification model.

[0029] like Figure 5 As shown in the experimental results, the text generation results indicate that, within the ERNIE-GEN-based text generation framework, Rouge-1 and Rouge-2 models using ELECTRA-small and ERNIE-tiny as fine-tuning frameworks achieved the lowest scores, while Rouge-1 and Rouge-2 models using RoBERTa as the fine-tuning framework achieved the highest scores. Therefore, RoBERTa performs best in both the generation and sentiment classification tasks.

[0030] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A public opinion guidance method based on an automatic generation model of public opinion sentiment text, characterized in that, include: The training model for automatically generating sentiment text in public opinion includes: The public opinion text data undergoes noise reduction preprocessing; a sentiment analysis module is trained, using a sentiment classifier to categorize the noise-reduced public opinion texts into three sentiment categories: positive, neutral, and negative, and public opinion texts in these categories; a topic extraction module is trained, using the TextRank algorithm to extract topics from the positive, neutral, and negative sentiment texts, resulting in positive sentiment topic P and negative sentiment topic N. The algorithm is as follows: in, Representative sentence The weights are given, and the summation on the right represents the contribution of each adjacent sentence to this sentence. Indicates the similarity between two sentences. Represents the sentence generated in the last iteration. The weights; This is the damping coefficient, with a value of 0.85; Train the public opinion text generation module by inputting the topic and corresponding sentiment text into the text generation model to obtain positive, neutral and negative sentiment public opinion text generators. Text is generated using the ERNIE-GEN model, the training of which includes: Supervised training is performed on a large amount of corpus data used for pre-training the RoBERTa model. Multi-granularity target sampling is combined to construct pre-training data, thereby enhancing the correlation between the encoder and decoder. The difference between training and inference is compensated by filling in the generation mechanism and noise-aware generation method; Using a multi-stream attention mechanism, a span-by-span generation stream is added to the word-by-word generation stream, allowing the model to generate a semantically complete segment each time. The formula for generating the word-by-word stream: Where Q, K, and V represent the query, key, and value in MultiHeadAttention, MH-Attn. It is the first Noisy target sequences in layers, Refers to the first Layer source sequence, Indicates the first The first layer There are two vector representations, where ai represents the i-th vector representation of the artificial symbol sequence AW in the l-th layer. The formula for generating flow across spans: ， in, Represents the i-th span, where , Indicates the first A span Vector representation; Training a sentiment classifier using the RoBERTa model includes: A deep learning model was used to annotate hundreds of thousands of public opinion text data to obtain an annotated dataset. A softmax neural network model was used to train sentiment classifiers for three categories of text: positive, neutral, and negative. Sentiment classification was performed on social network public opinion texts, resulting in positive, neutral, and negative texts. Generation of positive public opinion texts: Inputting negative public opinion topics into a positive public opinion text generator trained by an automatic public opinion sentiment text generation model can quickly generate a large number of positive public opinion texts corresponding to negative public opinion topics, which can be used to guide online public opinion. Guiding online public opinion: Automatically feed positive public opinion texts into online platforms to divert attention from current conflicts and guide public opinion.

2. The public opinion guidance method as described in claim 1, characterized in that, The training of the public opinion text production model includes: Positive sentiment keywords extracted by the Textrank algorithm are used as labels, and positive sentiment blog posts are used as text input for training the ERNIE-GEN model; ERNIE-GEN was trained using a positive sentiment generation dataset to obtain a positive sentiment text generator; After sentiment classification and theme extraction, the public opinion theme is modeled. Input negative public opinion topics into the positive text generator to obtain generated positive text.

3. A network public opinion guidance device based on the ERNIE-GEN model for implementing the public opinion guidance method as described in any one of claims 1-2, characterized in that, include: The ERNIE-GEN model training module is used to denoise the topic information and comment information of public opinion on online social platforms, use the RoBERTa sentiment classifier to divide the public opinion text into different sentiment categories, use the TextRank algorithm to calculate the sentiment text, extract the sentiment topic words, and input them into the specified model training module to obtain different sentiment public opinion text generators, which can be called by the positive public opinion text generation module. The positive public opinion text generation module uses the ERNIE-GEN model to generate a specified number of positive public opinion texts, and then stores and sends these texts to the positive public opinion text delivery module. The positive public opinion text delivery module is used to call or simulate the public opinion release function of the corresponding public opinion platform based on the identity key of each important public opinion platform filled in by the user, and release the generated positive public opinion text, thereby guiding negative public opinion and diverting public attention.

Images