System for detecting sentiment in social media based on pe-m-bert knowledge embeddings

Abstract

The application relates to a knowledge embedding social media emotion detection system based on PeM-Bert, and relates to the field of text detection. The application is aimed at solving the problems of weak text emotion recognition pertinence and poor emotion classification granularity of existing social media text emotion recognition methods. The application comprises the following steps: obtaining an original emotion text sequence X, setting emotion labels for the original emotion text sequence X, using the original emotion text with the set emotion labels to form an original data set, and dividing the original data set into a first training set and a first test set; training an attention mechanism model using the first training set to obtain a trained attention mechanism model; testing the trained attention mechanism model using the first test set, and taking the trained attention mechanism model that passes the test as an emotion detection network; inputting a text sequence to be detected into the emotion detection network to obtain an emotion detection result. The application is used for emotion detection of social media text.

Description

Technical Field

[0001] This invention relates to the field of text detection, and in particular to a knowledge-embedded social media sentiment detection system based on PeM-Bert. Background Technology

[0002] With the rapid development of online social networking platforms (such as Twitter and Facebook), social networks have become an integral part of people's daily lives. Various websites are seizing this opportunity to conduct sentiment analysis on massive amounts of data to launch new features and attract new customers. Online social media (OSM) platforms provide opportunities to express, communicate, and share people's opinions, thoughts, viewpoints, and perspectives on local and international issues, matters, and topics through text, images, audio, and video posts. These social media posts are public and emotionally charged, so analyzing and studying them can reveal emotional states and the reasons behind these emotions. Understanding these emotions is crucial for system improvement. Therefore, all these social platforms play a leading role in investigating social trends, tracking customer feedback, and understanding and developing business strategies to help customers / consumers make decisions by analyzing people's emotions and feelings. Therefore, identifying and analyzing emotions in social media is of great significance.

[0003] Traditional social media text sentiment recognition methods include keyword-based, corpus-based, and rule-based approaches. However, these methods struggle to handle long-term dependencies in social media text. With the development of deep learning methods, a deep architecture based on neural attention, called Transformer, has been proposed. Transformers excel at handling long-term dependencies in social media text. Since the introduction of the initial Transformer model, various language models have been proposed, such as BERT, XLNet, RoBERTa, and ALBERT. These models have achieved good results in sentiment detection for social media text. However, they do not consider downstream tasks, resulting in weak targeting of sentiment recognition and thus low accuracy. Furthermore, current social media text sentiment recognition methods focus on identifying six "basic" emotions (happiness, sadness, anger, disgust, surprise, and fear), but fail to capture the broader range of emotions people experience and express in daily life, such as more granular emotions like sadness, excitement, optimism, and despair. This leads to poor granularity in the identified emotion classifications. Summary of the Invention

[0004] The purpose of this invention is to address the problems of weak targeting of text emotion recognition and poor granularity of emotion classification in existing social media text emotion recognition methods, and to propose a social media emotion detection system based on PeM-Bert knowledge embedding.

[0005] The PeM-Bert-based knowledge-embedded social media sentiment detection system includes: a sequence acquisition module, a sentiment detection module, and a result output module.

[0006] The sequence acquisition module is used to acquire the text sequence of the emotion to be detected and send the text sequence of the emotion to be detected to the emotion detection module.

[0007] The emotion detection module is used to input the text sequence of the emotion to be detected into the emotion detection network, obtain the emotion detection result, and input the emotion detection result into the result output module.

[0008] The result output module is used to output the emotion detection results;

[0009] The emotion detection network was obtained through the following methods:

[0010] Step 1: Obtain the original emotional text sequence X, assign emotional labels to the original emotional text sequence X, use the original emotional texts with assigned emotional labels to form the original dataset, and divide the original dataset into the first training set and the first test set.

[0011] Step 2: Train the attention mechanism model using the first training set to obtain a trained attention mechanism model;

[0012] Step 3: Test the trained attention mechanism model using the first test set. If the accuracy of the trained attention mechanism model is greater than or equal to the first preset threshold, then use the trained attention mechanism model as the emotion detection network. If the accuracy of the trained attention mechanism model is less than the first preset threshold, then return to step 1.

[0013] Furthermore, the emotional labels include: admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disgust, joy, disappointment, disapproval, embarrassment, excitement, fear, gratitude, grief, love, nervousness, optimism, pride, realization, relief, remorse, sadness, surprise, and neutral.

[0014] Furthermore, the attention mechanism model includes: a knowledge base, a representation matrix acquisition layer, a dense layer, an attention layer, and a two-layer dense network;

[0015] The knowledge base is used to convert the original emotional text sequences in the first training set into feature vectors X. e and the feature vector X e Send to the dense layer;

[0016] The representation matrix acquisition layer uses the original emotional text sequence X and the pre-trained PeM-Bert model to obtain the representation matrix T corresponding to the hidden state of the last layer of the pre-trained PeM-Bert model. c ={t0,...,t N}, and T c Send to the attention layer;

[0017] Among them, T c ∈R N′lc , t b It is a contextual representation of the original emotion sequence, where b ranges from 0 to N, N is the total number of time steps, and l c It is the size of the output representation generated by the pre-trained PeM-Bert model;

[0018] The dense layer is used to process the feature vector X. e Projection yields sentence-level sentiment encoding H e ∈R le′lc and H e Send to the attention layer;

[0019] Among them, l e It is X e The dimension;

[0020] The attention layer utilizes Tc and H e Obtain the attention score s, then use the attention score s to obtain the final text representation, and send the final text representation to the two-layer dense network;

[0021] The two-layer dense network performs emotion classification based on the final text representation to obtain the final emotion category.

[0022] Furthermore, the pre-trained PeM-Bert model is obtained in the following way:

[0023] A1. Preprocess the original emotional text sequence X to obtain the preprocessed original emotional text sequence S;

[0024] A2. Use S with emotion labels as the pre-training dataset, and divide the pre-training dataset into a second training set and a second test set;

[0025] A3. Use the second training set to pre-train the PeM-Bert neural network model to obtain the pre-trained PeM-Bert neural network model;

[0026] A4. Test the pre-trained PeM-Bert neural network model using the second test set. If the accuracy of the pre-trained PeM-Bert neural network model is greater than or equal to the preset second threshold, then use the pre-trained PeM-Bert neural network model as the pre-trained PeM-Bert model. If the accuracy of the trained PeM-Bert neural network model is less than the preset second threshold, then return to A1.

[0027] Further, the preprocessing of the original emotional text sequence X in A1 to obtain the preprocessed original emotional text sequence S is specifically as follows:

[0028] A11. Multiply the length of the original emotional text sequence by a random number a to obtain the number of insertion marks;

[0029] Where a is a random number between 1 and 1 / 3;

[0030] A12. Randomly select punctuation marks from the punctuation set, and randomly insert punctuation marks into the original emotional text sequence according to the number of insertion marks in A11, to obtain the preprocessed original emotional text sequence S = [P0],...,[P i ],[P i+1 ...,[P m ];

[0031] The set of punctuation marks is {".",";","?",":","!","};

[0032] Among them, [P i] is the i-th emotional text block obtained by inserting punctuation marks into the original emotional text sequence, where m is the number of emotional text blocks.

[0033] Furthermore, the PeM-Bert neural network model includes: an encoder and a BERT model;

[0034] The encoder uses the P-tuning method to convert the preprocessed original emotional text sequence S into a template T:

[0035] T = {h0,...,h} i ,z(x),h i+1 ,...,h m ,z(y)}

[0036] Among them, h i Z(x) is a trainable vector, representing the vector representation of the emotion contained in the preprocessed original emotional text sequence, and Z(y) represents the vector representation of the target emotion category.

[0037] The BERT model utilizes the eMLM method and template T to optimize the encoder's output vector h. i .

[0038] Furthermore, the output vector h of the encoder is optimized using the eMLM method and template T. i Specifically:

[0039]

[0040] Where M(x,y) is the masking matrix. It is the downstream task loss function. It is {h0,...,h} i ,h i+1 ,...,h m The estimate of};

[0041] In the masking matrix M(x,y), the corresponding position of the template T part is 1, and the rest is 0.

[0042] Furthermore, the eMLM method masks the available dictionary in S. The probability P(w) of the word n ), as shown in the following formula:

[0043]

[0044] Where E is a dictionary that is available in S. The word 'k' is a dictionary entry that can be used to block certain words. The probability of the word, |·| represents the size of the set, w n S is not a part of the available dictionary. The word.

[0045] Furthermore, the attention layer utilizes T c and H e Obtain the attention score s, and then use the attention score s to obtain the final text representation, specifically as follows:

[0046] First, put T c and H e The connection is obtained by connecting the representations K∈R (N+le)′lc :

[0047] K = concat(T) c H e )

[0048] Then, the attention score s is obtained using K and t0:

[0049]

[0050] Finally, the attention score s is used to obtain the final text representation t. l :

[0051] t l =s T ·K.

[0052] Furthermore, the two-layer dense network is trained using the following loss function:

[0053]

[0054] Among them, Ω neg It is a set of non-target class samples, Ω pos It is a set of target class samples, r0 is the preset score of class 0, i' is the label of non-target class sample, j' is the label of target class sample, and r i' The score for non-target class samples, r j' It is the score of the target class sample.

[0055] The beneficial effects of this invention are as follows:

[0056] This invention introduces emotion masking language modeling in the pre-training model stage and utilizes the P-tuning method to enhance its adaptability to downstream task features. In the data preprocessing stage, it proposes data augmentation techniques and then uses sentence-level knowledge-embedded attention mechanism for emotion encoding to enrich the contextual representation, thus providing a richer representation of the input text. The contextual representation generated by the pre-trained PeM-Bert language model is then concatenated with the emotion encoding. This invention uses a softmax + cross-entropy loss function to address the class imbalance problem during emotion detection. This invention is highly targeted at text emotion recognition, identifying more nuanced emotions and improving the fine-grainedness of emotion classification. Attached Figure Description

[0057] Figure 1 The model is a PeM-Bert neural network.

[0058] Figure 2 This is a model structure for a knowledge-embedded attention mechanism. Detailed Implementation

[0059] Specific implementation method one: as follows Figure 2 As shown, this embodiment is a social media sentiment detection system based on PeM-Bert knowledge embedding, including: a sequence acquisition module, a sentiment detection module, and a result output module;

[0060] The sequence acquisition module is used to acquire the text sequence of the emotion to be detected and send the text sequence of the emotion to be detected to the emotion detection module.

[0061] The emotion detection module is used to input the text sequence of the emotion to be detected into the emotion detection network, obtain the emotion detection result, and input the emotion detection result into the result output module.

[0062] The result output module is used to output the emotion detection results;

[0063] The emotion detection network was obtained through the following methods:

[0064] Step 1: Obtain the original emotional text sequence X, assign an emotional label to the original emotional text sequence X, combine the emotional label and the original emotional text X to form the original dataset, and divide the original dataset into the first training set and the first test set.

[0065] The emotion labels include: admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disgust, joy, disappointment, disapproval, embarrassment, excitement, fear, gratitude, grief, love, nervousness, optimism, pride, realization, relief, remorse, sadness, surprise, and neutral.

[0066] Neutral means any emotion other than those mentioned above.

[0067] Step 2: Train the attention mechanism model using the first training set to obtain a trained attention mechanism model;

[0068] The attention mechanism model includes: a knowledge base, a representation matrix acquisition layer, a dense layer, an attention layer, and a two-layer dense network;

[0069] The knowledge base is used to convert the original emotional text sequence X in the first training set into a feature vector X. e ;

[0070] The representation matrix acquisition layer uses the original emotional text sequence X from the first training set and the pre-trained PeM-Bert model to obtain the representation matrix T corresponding to the hidden state of the last layer of the pre-trained PeM-Bert model. c ={t0,...,t N};

[0071] Among them, T c ∈R N′lc , t b This is the representation corresponding to the [CLS] label, which is the contextual representation of the original emotion sequence. b takes values ​​from 0 to N, where N is the total number of time steps.

[0072] The dense layer is used to process the feature vector X. e Projection yields sentence-level sentiment encoding H e ∈R le′lc ;

[0073] Among them, l e It is X e The dimension, l c This refers to the size of the output representation generated by the pre-trained PeM-Bert model, which is the dimension or length of the output representation vector, and this length is 768.

[0074] The attention layer utilizes T c and H e Obtain the attention score s, and use the attention score s to obtain the final text representation, specifically as follows:

[0075] First, put T c and H e The connection is obtained by connecting the representations K∈R (N+le)′lc :

[0076] K = concat(T) c H e )

[0077] Then, using the contextual representation t0 of the original emotion sequence as the query, the attention score s is obtained using K and t0:

[0078]

[0079] Finally, the attention score s is used to obtain the final text representation:

[0080] t l =s T ·K

[0081] The two-layer dense network uses the final text representation to classify emotions and obtain the final emotion category.

[0082] Step 3: Test the trained attention mechanism model using the first test set. If the accuracy of the trained attention mechanism model is greater than or equal to the first preset threshold, then use the trained attention mechanism model as the emotion detection network. If the accuracy of the trained attention mechanism model is less than the first preset threshold, then return to step 1.

[0083] In this implementation, the representations corresponding to the hidden states from the last layer of the PeM-Bert model are used to form a matrix T. c ={t0,...,t N The representation t0 corresponding to the [CLS] token is used as the context representation of the entire input text for classification. The input X is transformed into a feature vector X using a knowledge base. e To obtain the emotion encoding, its dimension depends on the dictionary data used, which we represent as l e Then, we project the transformed input vector using a dense layer to form a sentence-level sentiment code H. e H e ∈R le′lc Then, using self-attention techniques, emotions were encoded into H... e and context representation T c Connect them to form K, where K∈R (N+le)′lc We use t0 as the query to obtain the softmax-attention score s. The final representation tl is obtained by weighting matrix K with s. H e With T c Including them together in K helps preserve the contextual information learned by the encoder, as well as the added emotional knowledge, while reweighting t0.

[0084] Specific implementation method two: such as Figure 1 As shown, the pre-trained PeM-Bert model is obtained in the following way:

[0085] A1. Preprocess the original emotional text sequence X to obtain the preprocessed original emotional text sequence S, specifically:

[0086] A11. Multiply the length of the original emotional text sequence by a random number a to obtain the number of insertion marks;

[0087] Where a is a random number between 1 and 1 / 3;

[0088] A12. Randomly select punctuation marks from the punctuation set, and randomly insert punctuation marks into the original emotional text sequence according to the number of insertion marks in A11, to obtain the preprocessed original emotional text sequence S = [P0],...,[P i ],[P i+1 ...,[P m ];

[0089] The punctuation set is {".",";","?",":","!",","}, [P i ] represents the i-th emotional text block obtained by inserting punctuation marks into the original emotional text sequence, where m is the number of emotional text blocks;

[0090] A2. Combine the preprocessed original sentiment text sequence S and the labels to form a pre-training dataset, and divide the pre-training dataset into a second training set and a second test set.

[0091] A3. Use the second training set to pre-train the PeM-Bert neural network model to obtain the pre-trained PeM-Bert neural network model;

[0092] The PeM-Bert neural network model includes: an encoder and a BERT model;

[0093] The encoder uses the P-tuning method to convert the preprocessed original emotional text sequence S into a template T, specifically:

[0094] T = {h0,...,h} i ,z(x),h i+1 ,...,h m ,z(y)}

[0095] Among them, h i (0≤i<m) are trainable vectors S=[P0],...,[P i ],[P i+1 ...,[P m The input is the vector obtained after encoding, z(x) represents the vector representation of the emotion related to the input text, and z(y) represents the vector representation of the target or reference emotion category.

[0096] The BERT model uses the eMLM method to optimize the trainable vector h using a template T. i As shown in the following formula:

[0097]

[0098] Where M(x,y) is the masking matrix. It is the downstream task loss function. It is {h0,...,h} i ,h i+1 ,...,h m The estimate of};

[0099] In the masking matrix M(x,y), the corresponding position of the template T is 1, and the rest is 0;

[0100] The masking dictionary obtained using the eMLM method belongs to the available dictionary. The probability P(w) of the word n ), as shown in the following formula:

[0101]

[0102] Where E is a word in S that belongs to the available dictionary, k is the probability of masking words in the available dictionary, |·| represents the size of the set, and w n It is a word in S that is not in the available dictionary;

[0103] A4. Test the pre-trained PeM-Bert neural network model using the second test set. If the accuracy of the pre-trained PeM-Bert neural network model is greater than or equal to the preset second threshold, then use the pre-trained PeM-Bert neural network model as the pre-trained PeM-Bert model. If the accuracy of the trained PeM-Bert neural network model is less than the preset second threshold, then return to step A1.

[0104] This implementation differs from the traditional Prompt; P-tuning uses a pseudo-[P]... iThe discrete tokens are replaced with [], and then their continuous vectors are used to construct the target, i.e., the template T. P-tuning does not randomly initialize several new tokens and then train directly. Instead, it uses a small LSTM model to calculate these embeddings and makes this LSTM model learnable. However, adding an LSTM feels a bit awkward and is slightly more complicated to implement. Therefore, in the data construction of this invention, for the training set, a mask is constructed according to the proportion of the pre-trained eMLM task, and no new mask is constructed for the validation and test sets. In the training part, this invention freezes all gradients of the pre-trained PeM-Bert model except for the embedding layer; and for the backpropagation of the embedding layer, this invention designs a mask matrix in which only the corresponding position of the template part is 1 and the other parts are 0. By register_hook, the gradient is updated only when the gradient is updated in each propagation, so as to only update the embedding of the template part. Unlike BERT, which uses a uniform probability (15%) to mask the tokens in the input sentence, we assign a higher probability to the available dictionary. We extract words with rich emotional content from the dictionary. We use k to represent this probability, which is a hyperparameter in the eMLM method. The masking process can be summarized as follows: Given an input sentence S: extract words belonging to the dictionary... The words are denoted by E; the masking probability of these words is set to 1. This ensured that a total of 15% of words were masked. This invention uses P-tuning to fine-tune the entire model, including the eMLM embedding layer and other layers. This combined approach fully leverages the prior knowledge extracted by eMLM while taking full advantage of the fine-tuning capabilities of P-tuning, thereby improving the model's performance in sentiment detection.

[0105] Specific implementation method three: The loss function used to train the two-layer dense network is obtained in the following way:

[0106] Finally, during the classification, t l The input is fed into a two-layer dense network to obtain the corresponding emotion output probability. During the final classification, we need to select f target categories from n' candidate categories. A common approach is to use sigmoid activation, turning it into n binary classification problems, and using the sum of the cross-entropy of the binary classifications as the loss. Clearly, when n >> f, this approach faces a severe class imbalance problem.

[0107] The cross-entropy for single-label classification is shown in the following formula:

[0108]

[0109] Where, {s1,...,s t-1 ,s t+1 ,...,s n'} represents the non-target class score, s t It is the target class score, s c The score is for the non-target class, and c takes the form {1,...t-1,t+1,...,n'}.

[0110] Having previously learned that logsumexp is actually a smooth approximation of max, we have the following formula:

[0111]

[0112] The characteristic of this loss is that all non-target class scores {s1,...,s} t-1 ,s t+1 ,...,s n'} and target class score {s t By comparing the differences between each pair of classes, the maximum value of the difference should be as small as possible, thus achieving the effect that "the score of the target class is greater than the score of each non-target class".

[0113] Therefore, in a multi-label classification scenario with multiple target classes, this invention aims to ensure that "the score for each target class is not less than the score for each non-target class," thus yielding the following loss function:

[0114]

[0115] Where Ω pos Ω neg These are the sets of positive and negative classes for the samples.

[0116] For multi-label classification where f is not fixed, a threshold is needed to determine which classes to output. Therefore, this invention also introduces an additional zero class, aiming for the target class scores to be greater than r0 and the non-target class scores to be less than r0. We hope r... i' <r j' Just add it to the log.

[0117] Therefore, the final Loss form obtained by the "softmax + cross-entropy" method is shown in the following formula:

[0118]

[0119] Among them, Ω neg It is the set of negative class samples, i.e., the set of samples that are not in the target class, Ω posIt is the set of positive class samples, i.e., the set of target class samples, where r0 is the preset score for class 0, i' is the label of the non-target class sample, j' is the label of the target class sample, and r i' The score for non-target class samples, r j' It is the score of the target class sample.

[0120] This invention avoids class imbalance because it doesn't transform multi-label classification into multiple binary classification problems. Instead, it compares the scores of the target class with those of non-target classes pairwise, automatically balancing the weights of each item thanks to the favorable properties of logsumexp. Therefore, in the final classification, this invention employs a "softmax + cross-entropy" scheme to obtain the probability of the corresponding input's emotional output.

Claims

1. A social media sentiment detection system based on PeM-Bert knowledge embedding, characterized in that... The system includes: a sequence acquisition module, an emotion detection module, and a result output module; The sequence acquisition module is used to acquire the text sequence of the emotion to be detected and send the text sequence of the emotion to be detected to the emotion detection module. The emotion detection module is used to input the text sequence of the emotion to be detected into the emotion detection network, obtain the emotion detection result, and input the emotion detection result into the result output module. The result output module is used to output the emotion detection results; The emotion detection network was obtained through the following methods: Step 1: Obtain the original emotional text sequence and the original emotional text sequence Set emotion labels, use the original emotion texts with set emotion labels to form the original dataset, and divide the original dataset into the first training set and the first test set; Step 2: Train the attention mechanism model using the first training set to obtain a trained attention mechanism model; The attention mechanism model includes: a knowledge base, a representation matrix acquisition layer, a dense layer, an attention layer, and a two-layer dense network; The knowledge base is used to convert the original emotional text sequences in the first training set into feature vectors. and the feature vector Send to the dense layer; The representation matrix acquisition layer utilizes the original emotional text sequence. Obtain the representation matrix corresponding to the hidden state of the last layer of the pre-trained PeM-Bert model. and will Send to the attention layer; in, , It is a contextual representation of the original emotional sequence. The time step ranges from 0 to N, where N is the total number of time steps. It is the size of the output representation generated by the pre-trained PeM-Bert model; The pre-trained PeM-Bert model is obtained through the following method: A1. The original emotional text sequence Perform preprocessing to obtain the preprocessed original emotional text sequence S; A2. Use S with emotion labels as the pre-training dataset, and divide the pre-training dataset into a second training set and a second test set; A3. Use the second training set to pre-train the PeM-Bert neural network model to obtain the pre-trained PeM-Bert neural network model; A4. Test the pre-trained PeM-Bert neural network model using the second test set. If the accuracy of the pre-trained PeM-Bert neural network model is greater than or equal to the preset second threshold, then use the pre-trained PeM-Bert neural network model as the pre-trained PeM-Bert model. If the accuracy of the trained PeM-Bert neural network model is less than the preset second threshold, then return to A1. The dense layer is used for feature vectors Projection to obtain sentence-level sentiment encoding and will Send to the attention layer; in, yes The dimension; The attention layer utilizes and Obtain attention score Thus utilizing attention scores Obtain the final text representation and send it to the two-layer dense network; The two-layer dense network performs emotion classification based on the final text representation to obtain the final emotion category. Step 3: Test the trained attention mechanism model using the first test set. If the accuracy of the trained attention mechanism model is greater than or equal to the first preset threshold, then use the trained attention mechanism model as the emotion detection network. If the accuracy of the trained attention mechanism model is less than the first preset threshold, then return to step 1.

2. The knowledge-embedded social media sentiment detection system based on PeM-Bert according to claim 1, characterized in that: The emotion labels include: admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disgust, joy, disappointment, disapproval, embarrassment, excitement, fear, gratitude, grief, love, nervousness, optimism, pride, realization, relief, remorse, sadness, surprise, and neutral.

3. The knowledge-embedded social media sentiment detection system based on PeM-Bert according to claim 2, characterized in that: The original emotional text sequence in A1 Preprocessing is performed to obtain the preprocessed original emotional text sequence S, specifically: A11. Multiply the length of the original emotional text sequence by a random number a to obtain the number of insertion marks; Where a is a random number between 1 and 1 / 3; A12. Randomly select punctuation marks from the punctuation set and randomly insert them into the original emotional text sequence according to the number of insertion marks in A11, to obtain the preprocessed original emotional text sequence. ; The set of punctuation marks is {".", ";", "?", ":", "!",","}; in, The i-th emotional text block is obtained by inserting punctuation marks into the original emotional text sequence, where m is the number of emotional text blocks.

4. The knowledge-embedded social media sentiment detection system based on PeM-Bert according to claim 3, characterized in that: The PeM-Bert neural network model includes: an encoder and a BERT model; The encoder uses the P-tuning method to convert the preprocessed original emotional text sequence S into a template T: in, It is a trainable vector, z(x) represents the vector representation of emotion contained in the preprocessed original emotional text sequence. A vector representation of the target emotion category; The BERT model utilizes the eMLM method and template T to optimize the encoder's output vector. .

5. The knowledge-embedded social media sentiment detection system based on PeM-Bert according to claim 4, characterized in that: The encoder output vector is optimized using the eMLM method and template T. Specifically: in, It is a shielding matrix. It is the downstream task loss function. yes The estimate; Shielding matrix The T part of the template corresponds to a 1, while the rest are 0.

6. The knowledge-embedded social media sentiment detection system based on PeM-Bert according to claim 5, characterized in that: The eMLM method masks words in the available dictionary in S. Probability of Chinese words As shown in the following formula: in, It belongs to the available dictionary in S. The word 'k' is a dictionary entry that can be used to block certain words. The probability of the word being in the middle. Indicates the size of the set. S is not a part of the available dictionary. The word.

7. The knowledge-embedded social media sentiment detection system based on PeM-Bert according to claim 6, characterized in that: The attention layer utilizes and Obtain attention score Thus utilizing attention scores To obtain the final text representation, specifically: First, and Connection obtained connection representation : Then, using and Obtain attention score : Finally, using attention scores Obtain the final text representation : 。 8. The knowledge-embedded social media sentiment detection system based on PeM-Bert according to claim 7, characterized in that: The two-layer dense network is trained using the following loss function: in, It is a collection of samples that are not part of the target class. It is a collection of target class samples. It is a preset score for category 0. These are non-target class sample labels. It is the target class sample label. The score is for non-target class samples. It is the score of the target class sample.

Images