Environment petition emotion analysis method and system based on improved Bert model

Abstract

The invention discloses an environment petition emotion analysis method and system based on an improved Bert model, and the method comprises the steps: collecting environment petition complaint text data, carrying out the data cleaning and labeling, obtaining an emotion classification and speaking detection data set, and constructing a speaking keyword lexicon; a pre-trained Bert model is adopted as a basis, a bidirectional long-short-term memory network is combined to capture a text long-short-term context dependency relationship, a post-self-attention mechanism is introduced to enhance the recognition capability of key emotion information and speaking features, keyword matching is utilized to assist speaking detection, and the misjudgment rate is effectively reduced; and finally, deploying the trained model in an online intelligent petition system, automatically analyzing an emotion level and a speaking risk of a complaint text in real time, effectively identifying a high-risk event, improving the automation and intelligence level of environmental petition management, and improving the efficiency of environmental emergency management.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing and sentiment analysis, and in particular to an environmental petition sentiment analysis method and system based on an improved Bert model. Background Art

[0003] Traditional environmental petition management methods usually rely on manual processing, which has problems such as long time consumption, low efficiency, and strong subjectivity. At the same time, most existing complaint text sentiment analysis methods use traditional machine learning or simple deep learning models, and cannot deeply mine and understand the sentiment information and potential risks in complaint texts, especially in the aspect of handling the identification of threatening information, there is a problem of insufficient accuracy.

[0004] In recent years, pre-trained language models represented by the Bert model have made remarkable progress in the field of natural language processing, and can efficiently capture the semantic information and context relationship of complaint texts. However, when the existing Bert model is directly used for the sentiment analysis of petition complaint texts, there are still deficiencies in accurately identifying complex sentiment levels and threatening risks, resulting in the need to improve the accuracy of sentiment level classification and threatening detection.

[0005] Therefore, there is an urgent need to develop an improved Bert model to achieve more accurate and intelligent environmental petition sentiment and threatening risk analysis, thereby improving the intelligent level and decision-making efficiency of environmental petition management. Summary of the Invention

[0006] The first object of the present invention is to overcome the disadvantages and deficiencies of the prior art, and propose an environmental petition sentiment analysis method based on an improved Bert model, which can effectively solve the problem of low efficiency of traditional petition sentiment analysis means, realize the automated and intelligent processing of environmental petition sentiment and threatening risk analysis, and improve the accuracy and real-time response ability of environmental petition management.

[0007] The second object of the present invention is to provide an environmental petition sentiment analysis system based on an improved Bert model.

[0008] The third object of the present invention is to provide a storage medium.

[0009] The fourth object of the present invention is to provide a computing device.

[0010] The first object of the present invention is achieved by the following technical solutions: An environmental petition sentiment analysis method based on an improved Bert model, including the following steps:

[0011] 1) Obtain the complaint texts of users regarding environmental pollution, analyze the complaint texts, define multiple emotion levels and whether there is a threat, where a threat means whether the user will cause further complaints or create bad events, and establish a threat keyword library for assisting in threat detection judgment; perform data cleaning on the complaint texts to obtain an emotion classification dataset and a threat detection dataset respectively;

[0012] 2) Use the emotion classification dataset and the threat detection dataset to train an improved Bert model to obtain an optimal model; specifically, the improvement of this improved Bert model is: introduce a bidirectional long short-term memory network BiLSTM, a post self-attention mechanism and a residual connection after its backbone network to capture the emotion information and threat features in the complaint statements, and finally use a classifier to calculate the class probabilities; in addition, for the threat detection task, add a threat keyword detection branch, which is connected from the complaint text input to after the classifier output to perform threat keyword library matching detection on the complaint text;

[0013] 3) Deploy the optimal model to the online intelligent petition system, input the complaint text into the optimal model, and determine the emotion level of the complaint text and whether there is a threat according to the emotion level and threat determination result output by the optimal model, so as to realize the intelligent analysis of environmental complaint emotions.

[0014] Furthermore, step 1) includes the following steps:

[0015] 1.1) Data collection: Export the complaint texts of users regarding environmental pollution on the online petition platform, analyze the frequently occurring threat keywords, and establish a threat keyword library;

[0016] 1.2) Data cleaning: Perform deduplication on the obtained complaint texts, identify and remove exactly the same complaint texts, meaningless or worthless complaint texts in terms of actual demands, as well as text information lacking clear complaint content or obviously belonging to non-complaint categories, so as to avoid the influence of duplicate data and meaningless data on the model training process;

[0017] 1.3) Complaint text annotation: Label the cleaned data according to the emotion classification task and the threat detection task respectively; for the emotion classification task, the complaint texts are divided into three categories: mild dissatisfaction, moderate dissatisfaction and strong dissatisfaction; for the threat detection task, the complaint texts are divided into two categories: threat category and non-threat category. After annotation, an emotion classification dataset and a threat detection dataset are obtained;

[0018] 1.4) Dataset division: After the complaint text annotation is completed, divide the emotion classification dataset and the threat detection dataset into training sets, validation sets and test sets according to a preset ratio.

[0019] Further, in step 2), the improved Bert model is trained using a supervised learning method. Using the emotion classification dataset and the threat detection dataset as training samples, training is performed separately for the emotion classification task and the threat detection task. The input of the model is the complaint text, and the outputs are the emotion level of the complaint text and the threat determination result, respectively;

[0020] Among them, the improved Bert model includes: a backbone network, a bidirectional long short-term memory network BiLSTM, a post self-attention mechanism, a residual connection, and a classifier; for the input complaint text represented as sentence X, it is specifically represented as:

[0021] X = {x1, x2,..., x i ,..., x n}

[0022] In the formula, x i ∈R d , X ∈ R n×d , R is a real number, x i represents the i-th word in the sentence, n is the sentence length, and d is the embedding dimension size;

[0023] The backbone network uses the bert-base-chinese pre-trained language model to extract the semantic features of the complaint text and obtain the complaint text embedding representation:

[0024] H = {h cls , h1, h2,..., h i ,..., h n}

[0025] In the formula, h i ∈R d , H ∈ R n×d , h i represents the embedding representation of x i , h cls represents the global feature representation added by the Bert model, and H represents the embedding representation of the entire sentence;

[0026] Secondly, a bidirectional long short-term memory network BiLSTM is added after the backbone network to extract the long-term and short-term emotion dependencies of the complaint text, so as to more accurately grasp the context position relationship of the emotion information and the threat features, as follows:

[0027] H′ = BiLSTM(H)

[0028] In the formula, H′ represents the feature representation calculated by the bidirectional long short-term memory network BiLSTM, and H′ ∈ R n×d; After the bidirectional long short-term memory network (BiLSTM), a post self-attention mechanism is introduced to enhance the recognition of the most critical emotional information and threat features in the complaint text, enabling the model to focus more on the important features that determine the emotional level and threat status, thereby improving the model's capture ability when facing different emotional levels and threat features. The operation is expressed as:

[0029] H att = Attention(H′)

[0030] In the formula, H att is the result of the post self-attention calculation, and H att ∈R n×d ;

[0031] Finally, after the backbone network and after the post self-attention mechanism, a residual connection is added. A classifier composed of a fully connected layer and a Softmax function is used to perform classification predictions for emotion classification and threat detection respectively, as follows:

[0032] h out = mean(H att + H)

[0033] y e = Softmax(W e ·h out + b e )

[0034] y t = Softmax(W t ·h out + b t )

[0035] In the formula, h out is the global feature representation obtained by average pooling the sum of H att and H. The addition of H att and H represents the residual connection. W e , b e and W t , b t are the parameters of the fully connected layers for the two tasks of emotion classification and threat detection respectively. y e is the probability of the obtained emotional level, and y t is the probability of whether there is a threat. The level with the highest probability in y e is the emotional level of the complaint text;

[0036] For the threat detection task, while the complaint text is input into the model, the threat keyword matching method is used to scan the complaint text to find out whether there are threat keywords in the complaint text. The specific process is as follows: while the complaint text is input into the backbone network, the complaint text is input into the threat keyword detection branch, and word slicing processing is performed to obtain the word set W of the complaint text:

[0037] W = {w1, w2,..., w i′ ,..., w m}

[0038] In the formula, w i′ represents the i'-th word in the sentence, and m is the number of words in the complaint text; for each w i′ , the semantic similarity is calculated separately with each threat keyword in the threat keyword library. The similarity calculation model used is Word2Vec. A similarity threshold is set. If the similarity score between w i and any threat keyword in the threat keyword library reaches the set similarity threshold, then w i is determined to be a threat keyword. When the threat category probability output by the Softmax function corresponding to the threat detection is greater than the non-threat category probability, and a threat keyword is detected, the input complaint text is determined to be a threat, otherwise the input complaint text is determined to be non-threat.

[0039] Furthermore, in step 3), the optimal model is deployed in the online intelligent petition system. The working steps of this optimal model are as follows: when a complaint text is input, the model calculates and outputs the emotion level probability of the complaint text, and marks the level with the highest probability as the emotion level corresponding to the complaint text; then, the model outputs the determination probability of whether the complaint text is a threat. In addition, the threat keyword detection branch is used to detect whether the complaint text contains threat keywords. When the output threat category probability is higher than the non-threat category probability and a threat keyword is detected, the complaint text is determined to be in the threat category.

[0040] The second object of the present invention is achieved through the following technical solutions: an environmental petition emotion analysis system based on an improved Bert model, used to implement the above-mentioned environmental petition emotion analysis method based on an improved Bert model, including:

[0041] A data acquisition and processing module, used to acquire complaint texts of users about environmental pollution, analyze the complaint texts and define multiple emotion levels as well as define whether it is a threat. Among them, a threat represents whether the user will cause further complaints or cause bad events, and a threat keyword library is established for auxiliary judgment of threat detection; data cleaning is performed on the complaint texts to obtain an emotion classification data set and a threat detection data set respectively;

[0042] Training module, which constructs an improved Bert model and uses an emotion classification dataset and a threat detection dataset to train the improved Bert model to obtain an optimal model; specifically, the improvement of the improved Bert model is as follows: a bidirectional long short-term memory network BiLSTM, a post self-attention mechanism, and a residual connection are introduced after its backbone network to capture the emotion information and threat features in the complaint statement, and finally a classifier is used to calculate the class probability; in addition, for the threat detection task, a threat keyword detection branch is added, which is connected from the input of the complaint text to the output of the classifier to perform threat keyword library matching detection on the complaint text.

[0043] Analysis module, which is used to deploy the optimal model to the online intelligent petition system, input the complaint text into the optimal model, and determine the emotion level of the complaint text and whether it is a threat according to the emotion level and threat determination result output by the optimal model, so as to realize the intelligent analysis of the emotion of environmental complaints.

[0044] The third object of the present invention is achieved by the following technical solution: a storage medium stores a program, and when the program is executed by a processor, the above-mentioned method for analyzing the emotion of environmental petitions based on an improved Bert model is realized.

[0045] The fourth object of the present invention is achieved by the following technical solution: a computing device includes a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the above-mentioned method for analyzing the emotion of environmental petitions based on an improved Bert model is realized.

[0046] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0047] 1. By improving the Bert model and combining the bidirectional long short-term memory network BiLSTM, the post self-attention mechanism, and the residual connection, the present invention can more accurately capture the long-term and short-term emotion dependence relationships and threat features in the complaint text, effectively improving the accuracy of emotion classification and threat detection.

[0048] 2. The present invention constructs a threat keyword library for the threat detection task and introduces a threat keyword detection branch, further enhancing the ability to identify threat features and significantly reducing the risk of missed reports and false reports.

[0049] 3. The present invention can automatically and efficiently process a large amount of petition complaint text data, significantly reducing the need for manual intervention and greatly improving the efficiency and accuracy of environmental petition management.

[0050] 4. The model proposed by the present invention can be deployed in an online intelligent petition system, with real-time analysis and rapid response capabilities, which helps to timely discover and prevent environmental-related social risk events, and improve the level of environmental management decision-making and social stability. Description of the Drawings

[0051] Figure 1 It is a schematic diagram of the logical flow of the method of the present invention.

[0052] Figure 2 It is a schematic structural diagram of the improved Bert model; in the figure, E cls , E1, E2 to E n represent the initial embeddings of the complaint text, h cls , h1, h2 to h n are the embeddings extracted by the Bert model, BiLSTM is a bidirectional long short-term memory network, Self-Attention is a post self-attention mechanism, and the two Classifiers are the classifiers for the two tasks of emotion classification and threat detection respectively.

[0053] Figure 3 It is an architecture diagram of the system of the present invention. Detailed Embodiments

[0054] The present invention will be further described in detail below in conjunction with the embodiments and the drawings, but the embodiments of the present invention are not limited thereto.

[0055] Embodiment 1

[0056] As Figure 1 and Figure 2 shown, this embodiment discloses an environmental petition emotion analysis method based on an improved Bert model, and the specific situation is as follows:

[0057] 1) First, data collection and cleaning are performed, and environmental complaint text data submitted by users is collected from the online petition platform. First, strict preprocessing is performed on the collected data, including data deduplication operations to remove completely duplicate complaint contents; removing information that is obviously meaningless or has no actual claim value in the complaint text to ensure the high quality and effectiveness of the data; subsequently, professional annotators annotate the data for emotions and threats, where the emotion levels are divided into three categories: mild dissatisfaction, moderate dissatisfaction, and strong dissatisfaction, and the threat categories are divided into two categories: threat and non-threat. After annotation, the annotated data set is randomly divided into a training set, a validation set, and a test set according to a ratio of 8:1:1 for subsequent model training, validation, and testing.

[0058] Secondly, a threat keyword library is constructed. The construction of the threat keyword library is based on the characteristics of frequently occurring threat expressions in historical data, such as typical words that can reflect that the complainant may take further actions or cause bad events, such as "appeal", "exposure", "take extreme measures", etc., and they are summarized and sorted out. The obtained threat keyword library K = {k1, k2..., k i″ ,..., k m′, where m' is the size of the threat keyword library, and k i″ represents the i''-th threat keyword.

[0059] 2) Use the sentiment classification and threat detection dataset obtained in step 1) to train an improved Bert model. The specific improvement in the improved Bert model is as follows: The backbone network uses the Bert pre-trained model (Bidirectional Encoder Representations from Transformers). The Bert model is a bidirectional deep pre-trained language model based on the Transformer structure, composed of multiple Transformer encoder layers. Each encoder layer contains a multi-head self-attention mechanism and a feed-forward neural network. Different from traditional unidirectional or shallow models, Bert achieves deep context understanding through bidirectional joint training on a large amount of complaint text corpus. The Bert model has the ability to finely capture the complex context structure of language and is suitable for the deep semantic understanding and accurate classification tasks of the complex emotions and potential threat risks in the environmental petition complaint text of the present invention. The process is expressed as:

[0060] H = Bert(X)

[0061] X = {x1, x2,..., x i ,..., x n}

[0062] H = {h cls , h1, h2,..., h i ,..., h n}

[0063] In the formula, X represents the input complaint text, x i ∈R d , X ∈ R n×d , R is a real number, x i represents the i-th character in the sentence, n is the sentence length, and d is the embedding dimension size. H represents the embedding representation of the entire sentence, h i ∈R d , H ∈ R n×d , h i represents the embedding representation of x i , h clsRepresents the global feature representation added by the Bert model. Subsequently, in order to better capture the long-term and short-term dependencies between emotions and threatening features in complaint texts, a Bidirectional Long Short-Term Memory (BiLSTM) network is added after the output of the backbone network. The BiLSTM processes sequence information simultaneously from both the front and back directions of the complaint text through a bidirectional cyclic structure, effectively capturing the long-term context dependencies in the sentence and significantly improving the sensitivity to emotion features and threatening features in complaint texts. Specifically, the BiLSTM consists of two Long Short-Term Memory (LSTM) networks, a forward one and a backward one, which process the complaint text information from the starting position and the ending position of the complaint text sequence respectively. Through this bidirectional structure, the BiLSTM can consider the context information in both the front and back directions of the complaint text sequence simultaneously, significantly improving the model's ability to understand and remember complex semantic features in complaint texts, expressed as:

[0064] H′ = BiLSTM(H)

[0065] In the formula, H′ represents the feature representation calculated by the Bidirectional Long Short-Term Memory (BiLSTM) network, and H′ ∈ R n×d . The LSTM selectively retains or forgets historical information through gating units, including an input gate, a forget gate, and an output gate. The state transition and the output at each time step are expressed as:

[0066]

[0067] h t = o t ·tanh(C t )

[0068] In the formula, f t represents the forget gate, is the input gate representation, C t is the state representation at the current moment, C t-1 is the state representation at the previous time step, o t is the output gate representation, tanh is the activation function, and h t is the output of the LSTM at the current time step. Based on the BiLSTM, the present invention further introduces a post self-attention mechanism. The post self-attention mechanism effectively highlights the important features that determine the emotion level and threatening category in the complaint text by weighting and focusing on different parts of the input features, further enhancing the model's ability to capture and distinguish key features. This process is expressed as:

[0069] H att = Attention(H′)

[0070] where H att is the result of post self-attention calculation, and H att ∈R n×d . Specifically, the post self-attention mechanism first calculates three new vectors for the feature vectors of the model input: query (Q), key (K), and value (V). Each vector is obtained by multiplying the input feature by the corresponding weight matrix. Then, the attention scores are calculated using the dot product operation between the query vector and the key vector, and finally multiplied by the value vector to obtain the representation after attention. The formula is as follows:

[0071]

[0072] where d k is the dimension of the key vector, which is used to scale the dot product result to prevent the gradient instability problem caused by too large dot product. The attention scores are converted into a weight distribution through the Softmax function, which represents the degree of attention of the model to specific elements in the input sequence. Finally, the attention weights are applied to the value vector to achieve the adaptive weighted sum of the input features. In the present invention, the post self-attention mechanism is used to finely identify the particularly critical emotional features and threatening information in the complaint text. By dynamically adjusting the weights of each word or phrase in the input complaint text, the model can more accurately identify and focus on the key information that plays a decisive role in emotion classification and threat detection, effectively improving the model's ability to capture and classify complex emotions and threat risks.

[0073] A residual connection is introduced between the output of the backbone network and the post self-attention mechanism, and the output of the backbone network and the output of the post self-attention mechanism are added together to further optimize the gradient flow of the model, effectively alleviating the problems of gradient disappearance and gradient explosion during the training process, and enhancing the expression ability of the model.

[0074] Finally, the complaint text features processed by the above model structure are input into the classifier. The classifier uses a fully connected layer and the Softmax function to calculate the probability values of each category. The process is as follows:

[0075] h out = mean(H att + H)

[0076] y e = Softmax(W e ·h out + b e )

[0077] y t = Softmax(W t ·hout +b t )

[0078] In the formula, h out is the global feature representation obtained by performing mean pooling on the sum of H att and H. The addition of H att and H represents the residual connection. The role of the pooling operation is to reduce the feature dimension, extract and retain the most important and representative feature information in the input sequence, and at the same time reduce the sensitivity of the model to local noise or irrelevant information. W e , b e and W t , b t are the parameters of the fully connected layers for the two tasks of emotion classification and threat detection respectively. y e is the probability of the obtained emotion level, and y t is the probability of whether it is a threat. The level with the highest probability in y e is the emotion level of the complaint text.

[0079] Meanwhile, using the threat keyword library constructed in the early stage, the threat keyword detection method is used as an auxiliary detection means. The threat keyword library is obtained by mining and sorting out the high-frequency threat keywords in the threat information of historical environmental complaint texts, including typical threat expressions such as "appealing", "exposing", "taking extreme measures", etc., as well as related words and phrases that may attract public attention or have a greater social impact. After performing word segmentation on the input complaint text, the word segmentation result, that is, the word set W = {w1, w2,..., w i′ ,..., w m} is obtained, where w i′ represents the i'-th word in the sentence, and m is the number of words in the complaint text. Then, scan each word one by one and calculate the semantic similarity with all the threat keywords in the threat keyword library. Specifically, for each word w i′ , use Word2Vec to obtain the embedding features, and then use the cosine similarity to calculate the semantic similarity between this word and each threat keyword in the threat keyword library. The process is as follows:

[0080]

[0081]

[0082] In the formula, k j represents the j-th threat keyword, and represent the embedding features of w i′ and k j respectively, and S i′,jrepresents the semantic similarity of these two words. Set the similarity threshold to 0.8. If w i′ has a semantic similarity with the threatening keyword reaching the set similarity threshold, then it is determined that w i′ is a threatening keyword. To further assist in determining whether the complaint text belongs to the threatening category, when the probability of the threatening category output by the Softmax function corresponding to the threatening detection is higher than the probability of the non-threatening category and a threatening keyword is detected, the complaint text is determined to be in the threatening category; otherwise, the text is determined to be non-threatening. This method effectively improves the accuracy and recall rate of the threatening detection task and reduces the risks of missed reports and false alarms.

[0083] The present invention uses the PyTorch deep learning framework for model construction and training. The Adam optimizer is selected as the model optimizer to efficiently handle gradient updates and improve the stability of model training. The initial learning rate is set to 2e-5, and a learning rate scheduler is used for dynamic adjustment to achieve better convergence and model generalization performance during the training process. In terms of the loss function, the cross-entropy loss function (Cross Entropy Loss) is adopted and optimized for the emotion classification task and the threatening detection task respectively. By calculating the difference between the true label and the model's predicted output, it guides the update of the model parameters. After training is completed, the model parameters with the best performance are saved for subsequent deployment and use.

[0084] 3) Deploy the improved Bert model trained in step 2) to the online intelligent petition system. The Anaconda environment management is used for the model deployment environment. The PyTorch framework is used to load and run the trained model, and the trained improved Bert model is encapsulated as an API interface. When a new complaint text is input into the system, the system automatically transfers the complaint text to the Bert model API through the data interface for processing. If the emotion classification function is called, the improved Bert model calculates and outputs the probability of the emotion level of the complaint text, and marks the level with the highest probability as the emotion level corresponding to the complaint text. If the threatening detection function is called, in addition to the improved Bert model calculating and outputting the probability of whether it is a threat, a threatening keyword detection is also performed, and the two are combined to obtain the final result of whether it is a threat. The implementation method of the present invention effectively solves the problems of insufficient accuracy and low efficiency of traditional petition emotion analysis means. By combining model optimization with the threatening keyword detection branch, it realizes the automated and intelligent processing of environmental petition emotion and threatening risk analysis, and greatly improves the accuracy and real-time response ability of environmental petition management.

[0085] The following are the experimental results of the above-mentioned environmental petition emotion analysis method based on the improved Bert model in this embodiment:

[0086] For the emotion classification task, the accuracy, recall, F1-score, and relative accuracy are used to evaluate the precision of the improved model. Among them, relative accuracy is an indicator that can be used to evaluate the performance of the emotion classification task. It focuses on the impact of prediction errors between different categories on the task results. In this indicator, cross-category prediction errors are considered more serious than adjacent prediction errors. Its calculation formula is as follows:

[0087]

[0088] In the formula, N is the number of samples, is the predicted emotion category label, is the true emotion category label, is the label subscript. The comparison with the original Bert model was conducted in the form of ablation experiments, and the results are shown in Table 1 below.

[0089] Table 1

[0090]

[0091] The analysis of the experimental results shows that the improved Bert model performs significantly better than the original model in the emotion classification task. The accuracy has increased by 6.52 percentage points, the relative accuracy has increased by 3.60 percentage points, and the recall and F1-score have increased by 6.52 and 7.00 percentage points respectively. This proves that the improved model can more effectively distinguish emotion categories and make fewer serious cross-category prediction errors, verifying the significant advantages of the improved model.

[0092] For the threat detection task, the accuracy, precision, recall, F1-score, and relative accuracy are used to evaluate the precision of the improved model. The results are shown in Table 2 below.

[0093] Table 2

[0094]

[0095] The results of the threat detection experiment further show that the improved Bert model performs excellently. After introducing the threat keyword detection branch, the model accuracy reaches 99.51%, which is an increase of 22.88 percentage points compared to the original model. The precision is increased by 36.91 percentage points, the recall rate is increased by 11.73 percentage points, and the F1 value is significantly increased by 43.44 percentage points. This is because there are fewer examples of threats in the threat dataset, resulting in an unbalanced class distribution, making it difficult for the original model to accurately learn the characteristics of threats. However, introducing the threat keyword detection branch effectively alleviates the problem of insufficient samples and significantly enhances the model's ability to identify a small number of threat samples, thus greatly improving the performance of threat detection.

[0096] Example 2

[0097] This example discloses an environmental petition sentiment analysis system based on an improved Bert model, which is used to implement the environmental petition sentiment analysis method based on the improved Bert model described in Example 1, as Figure 3 shown, including the following functional modules:

[0098] The data acquisition and processing module is used to obtain the complaint text of users about environmental pollution, analyze the complaint text, define multiple emotion levels and define whether it is a threat. Here, a threat means whether the user will cause further complaints or create bad events, and establish a threat keyword library for auxiliary judgment of threat detection; perform data cleaning on the complaint text to obtain an emotion classification dataset and a threat detection dataset respectively;

[0099] The training module constructs an improved Bert model and uses the emotion classification dataset and the threat detection dataset to train the improved Bert model to obtain an optimal model; specifically, the improvement of this improved Bert model is: introduce a bidirectional long short-term memory network BiLSTM, a post self-attention mechanism and a residual connection after its backbone network to capture the emotion information and threat characteristics in the complaint statement, and finally use a classifier to calculate the class probability; in addition, for the threat detection task, add a threat keyword detection branch, which is connected from the complaint text input to the output of the classifier to perform threat keyword library matching detection on the complaint text;

[0100] The analysis module is used to deploy the optimal model to an online intelligent petition system, input the complaint text into the optimal model, and determine the emotion level of the complaint text and whether it is a threat according to the emotion level and threat judgment result output by the optimal model, so as to realize the intelligent analysis of environmental complaint emotions.

[0101] Example 3

[0102] This embodiment discloses a storage medium storing a program, which, when executed by a processor, implements the environmental petition sentiment analysis method based on the improved Bert model described in Embodiment 1.

[0103] The storage medium in this embodiment can be a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), a USB flash drive, a mobile hard disk, or other such media.

[0104] Embodiment 4

[0105] This embodiment discloses a computing device including a processor and a memory for storing the executable program of the processor. When the processor executes the program stored in the memory, it implements the environmental petition sentiment analysis method based on the improved Bert model described in Embodiment 1.

[0106] The computing device described in this embodiment can be a desktop computer, a laptop computer, a smart phone, a PDA handheld terminal, a tablet computer, a programmable logic controller (PLC), or other terminal devices with processor functions.

[0107] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. An environmental petition emotion analysis method based on an improved Bert model, characterized in that It includes the following steps: 1) Obtain the complaint text of users about environmental pollution, analyze the complaint text, define multiple emotion levels and define whether there is a threat. Here, a threat means whether the user will cause further complaints or create bad events, and establish a threat keyword library for assisting in threat detection judgment; clean the data of the complaint text to obtain an emotion classification data set and a threat detection data set respectively; 2) Use the emotion classification data set and the threat detection data set to train the improved Bert model to obtain the optimal model; specifically, the improvement of the improved Bert model is as follows: introduce a bidirectional long short-term memory network BiLSTM, a post self-attention mechanism and a residual connection after its backbone network to capture the emotion information and threat features in the complaint statement, and finally use a classifier to calculate the class probability; in addition, for the threat detection task, add a threat keyword detection branch, which is connected from the input of the complaint text to the output of the classifier to perform threat keyword library matching detection on the complaint text; 3) Deploy the optimal model to the online intelligent petition system, input the complaint text into the optimal model, and determine the emotion level of the complaint text and whether there is a threat according to the emotion level and threat judgment result output by the optimal model, so as to realize the intelligent analysis of environmental complaint emotions.

2. The environmental petition sentiment analysis method based on the improved Bert model according to claim 1, characterized in that, The step 1) includes the following steps: 1.1) Collect data: Export the complaint text of users about environmental pollution on the online petition platform, analyze the frequently occurring threat keywords, and establish a threat keyword library; 1.2) Data cleaning: Perform deduplication processing on the obtained complaint text, identify and remove completely identical complaint texts, meaningless or valueless complaint texts with no actual demands, and text information lacking clear complaint content or obviously belonging to non-complaint categories to avoid the influence of duplicate data and meaningless data on the model training process; 1.3) Complaint text annotation: Label the cleaned data according to the emotion classification task and the threat detection task respectively; for the emotion classification task, the complaint text is divided into three categories: mild dissatisfaction, moderate dissatisfaction and strong dissatisfaction; for the threat detection task, the complaint text is divided into two categories: threat category and non-threat category. After annotation, an emotion classification data set and a threat detection data set are obtained; 1.4) Data set division: After the complaint text annotation is completed, divide the emotion classification data set and the threat detection data set into training sets, validation sets and test sets according to a preset ratio respectively.

3. The method for analyzing environmental petition emotions based on the improved Bert model according to claim 1, characterized in that, In step 2), the improved Bert model is trained using the supervised learning method, using the emotion classification data set and the threat detection data set as training samples, and training is carried out for the emotion classification task and the threat detection task respectively. The input of the model is the complaint text, and the outputs are the emotion level of the complaint text and the threat judgment result respectively; Among them, the improved Bert model includes: a backbone network, a bidirectional long short-term memory network BiLSTM, a post self-attention mechanism, a residual connection and a classifier; for the input complaint text represented as sentence X, it is specifically represented as: X = {x1, x2,..., x i ,..., x n} where x i ∈R d , X ∈ R n×d , R is a real number, x i represents the i-th word in the sentence, n is the sentence length, and d is the embedding dimension size; The backbone network uses the bert-base-chinese pre-trained language model to extract the semantic features of the complaint text and obtain the embedded representation of the complaint text: H = {h cls , h1, h2, ..., h i , ..., h n} where h i ∈R d , H ∈ R n×d , h i represents the embedded representation of x i , h cls represents the global feature representation added by the Bert model, and H represents the embedded representation of the entire sentence; Secondly, a bidirectional long short-term memory network BiLSTM is added after the backbone network to extract the long-term and short-term emotional dependencies of the complaint text, so as to more accurately grasp the contextual position relationship between the emotional information and the threatening features, as follows: H′ = BiLSTM(H) where H′ represents the feature representation obtained by calculating through the bidirectional long short-term memory network BiLSTM, and H′ ∈ R n×d ; After the bidirectional long short-term memory network BiLSTM, a post self-attention mechanism is introduced to strengthen the recognition of the most critical emotional information and threat features in the complaint text, enabling the model to focus more on the important features that determine the emotional level and threat status, thereby improving the model's capture ability when facing different emotional levels and threat features. The operation is expressed as: H att = Attention(H′) where, H att is the result of the post self-attention calculation, and H att ∈R n×d ; Finally, a residual connection is added after the backbone network and after the self-attention mechanism. A classifier composed of a fully connected layer and a Softmax function is used to perform classification predictions on emotion classification and threat detection respectively, as follows: h out = mean(H att + H) y e = Softmax(W e ·h out + b e ) y t = Softmax(W t ·h out + b t ) where h out is the global feature representation obtained by average pooling the sum of H att and H, the addition of H att and H represents the residual connection, W e , b e and W t , b t are the parameters of the fully connected layers for the two tasks of emotion classification and threat detection respectively, y e is the probability of the obtained emotion level, y t is the probability of whether there is a threat, and the level with the highest probability in y e is the emotion level of the complaint text; For the threat detection task, while the complaint text is input into the model, the threat keyword matching method is used to scan the complaint text to find out whether there are threat keywords in the complaint text. The specific process is as follows: while the complaint text is input into the backbone network, the complaint text is input into the threat keyword detection branch to perform word slicing processing to obtain the word set W of the complaint text: W = {w1, w2,..., w i′ ,..., w m} where w i′ represents the i'-th word in the sentence, and m is the number of words in the complaint text; for each w i′ , semantic similarity calculation is performed separately with each threatening keyword in the threatening keyword library. The similarity calculation model used is Word2Vec, and a similarity threshold is set. If the similarity score between w i and any threatening keyword in the threatening keyword library reaches the set similarity threshold, then w i is determined to be a threatening keyword. When the probability of the threatening category output by the Softmax function corresponding to the threatening detection is greater than the probability of the non-threatening category, and a threatening keyword is detected, the input complaint text is determined to be threatening; otherwise, the input complaint text is determined to be non-threatening.

4. The environmental petition emotion analysis method based on the improved Bert model according to claim 1, characterized in that In step 3), the optimal model is deployed to the online intelligent petition system. The working steps of this optimal model are as follows: when a complaint text is input, the model calculates and outputs the emotion level probability of the complaint text, and marks the level with the highest probability value as the emotion level corresponding to the complaint text; then, the model outputs the judgment probability of whether the complaint text threatens. In addition, the threat keyword detection branch is used to detect whether the complaint text contains threat keywords. When the output threat category probability is higher than the non-threat category probability and threat keywords are detected, the complaint text is determined to be a threat category.

5. An environmental petition emotion analysis system based on an improved Bert model, characterized in that, An environmental petition emotion analysis method based on an improved Bert model according to any one of claims 1 to 4, comprising: A data acquisition and processing module, which is used to acquire the complaint text of users about environmental pollution, analyze the complaint text, define multiple emotion levels and define whether it threatens. Among them, threatening means whether the user will cause further complaints or cause bad events, and establish a threat keyword library for assisting in judging threat detection; perform data cleaning on the complaint text to obtain an emotion classification data set and a threat detection data set respectively; A training module, which constructs an improved Bert model and uses the emotion classification data set and the threat detection data set to train the improved Bert model to obtain an optimal model; among them, the specific improvement of this improved Bert model is: a bidirectional long short-term memory network BiLSTM, a post self-attention mechanism and a residual connection are introduced after its backbone network to capture the emotional information and threatening features in the complaint statement, and finally a classifier is used to calculate the category probability; in addition, for the threat detection task, a threat keyword detection branch is added, which is connected from the input of the complaint text to the output after the classifier to perform threat keyword library matching detection on the complaint text; An analysis module, which is used to deploy the optimal model to the online intelligent petition system, input the complaint text into the optimal model, and determine the emotion level of the complaint text and whether it threatens according to the emotion level and threat judgment result output by the optimal model, so as to realize the intelligent analysis of environmental complaint emotions.

6. A storage medium stores a program, characterized in that, When the program is executed by a processor, it implements the method for analyzing environmental petition emotions based on an improved Bert model according to any one of claims 1 to 4.

7. A computing device, comprising a processor and a memory for storing processor-executable programs, characterized in that, When the processor executes the program stored in the memory, it implements the method for analyzing environmental petition emotions based on an improved Bert model according to any one of claims 1 to 4.

Images