Dual sentiment classification method based on diffusion counterfactual data augmentation

Abstract

The present application relates to the field of natural language processing, and more particularly to a double sentiment classification method based on diffusion counterfactual data enhancement. In view of the problem that the existing technology improves the performance of the model by synonym sentence data enhancement or by adversarial perturbation on word embedding, but does not solve the problem of pseudo-association within the data, resulting in poor sentiment classification effect, a double sentiment classification method based on diffusion counterfactual data enhancement is proposed. A double sentiment classification model DCA based on diffusion counterfactual data enhancement is established and obtained; the sentiment data to be classified is input into the trained double sentiment classification model DCA based on diffusion counterfactual data enhancement, and double sentiment classification is completed. Two predictors are used as sentiment classifiers for double sentiment classification. By selecting the output result of the classifier with higher confidence, the problem of pseudo-association within the data in the prior art is solved, thereby improving the effect of sentiment classification.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and in particular to a dual sentiment classification method based on diffusion counterfactual data enhancement. Background Technology

[0002] With the widespread popularity of social media websites, news portals, and forums, people can freely and conveniently express their opinions. These views and opinions profoundly and truthfully reflect the subjective socio-political attitudes concerning people's lives, social production, and emotions, which are of great significance to research fields such as public opinion surveys, market trend analysis, recommendation systems for personalized user recommendations, public health monitoring, information retrieval, and rumor detection. Meanwhile, automatically extracting semantic information from natural language text has always been an important research problem with many practical applications, solving problems such as sentiment analysis, irony, controversy, sarcasm, rumor and fake news detection, stance detection, and argument mining. In recent years, in the field of sentiment analysis research, the use of pre-trained language models (such as BERT, GPT, XLNet, RoBERTa, ALBERT, T5, and ERNIE) has significantly improved the performance of natural language processing tasks. The inherent complexity of human emotions poses a continuous challenge in practical sentiment analysis, leading to problems such as overfitting. This challenge can cause model prediction failures when encountering minor modifications in real-world examples. Researchers have attempted to address these challenges using data augmentation and adversarial perturbations, such as enhancing training data by generating synonyms or introducing random noise during word embedding to improve model robustness. However, models trained on independently and identically distributed (IID) data show a declining generalization ability on out-of-domain (OOD) data. Therefore, improving the generalization ability of sentiment analysis models to meet the needs of complex real-world applications has become a significant and insurmountable research challenge.

[0003] Previous sentiment analysis research, aimed at enhancing the robustness of neural networks, has primarily fallen into three categories: adversarial training, causal inference, and data augmentation. Adversarial training introduces adversarial noise during training, adding regularization terms or employing exit strategies to help the model produce consistent outputs when faced with data perturbations. Causal inference aims to improve model robustness by identifying factors in text that truly influence sentiment; some studies emphasize establishing causal relationships between word features and sentiment labels. Data augmentation primarily focuses on improving the model's generalization ability by generating more training data. Some studies have proposed synonym-based augmentation, which randomly replaces corresponding words in the original samples with synonyms, hypersynonyms, or hyposynonyms. However, synonym augmentation methods cannot effectively address the problem of spurious associations in the original samples. To address this, some studies have made minimal modifications to the training data, using manual annotation to perform label inversion. However, this manual method is expensive and time-consuming. Furthermore, some studies have employed automation techniques, such as formulating antonym replacement rules to automatically generate antonym samples under those rules. This has alleviated the problem of spurious associations in the original data to some extent. However, current counterfactual methods have several limitations: (i) Most studies rely on fixed rules or known negation words, synonyms, and antonyms in WordNet to create antonym samples, limiting the diversity and semantic consistency of the generated samples. (ii) During training, the generated anonymous samples are merged with the original samples without considering their correspondence. (iii) Data generation and classification are usually treated as different tasks and trained sequentially, leading to problems such as error accumulation.

[0004] Therefore, current sentiment analysis methods suffer from poor performance in sentiment classification because the models are trained on in-domain (IID) data and cannot handle out-of-domain (OOD) data well. Furthermore, they often improve model performance by using synonym data augmentation or adversarial perturbation of word embeddings, without addressing the problem of spurious associations within the data. Summary of the Invention

[0005] The purpose of this invention is to address the problem in existing technologies that improve model performance through synonym data augmentation or adversarial perturbation of word embeddings, but fail to solve the problem of spurious associations within the data, resulting in poor sentiment classification. We propose a dual sentiment classification method based on diffusion counterfactual data augmentation.

[0006] I. Establish a dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation; obtain a trained dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation;

[0007] 2. Input the sentiment data to be classified into the trained dual sentiment classification model DCA based on diffusion counterfactual data augmentation to complete the dual sentiment classification.

[0008] The first step involves establishing a dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation; obtaining a trained dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation; the specific process is as follows:

[0009] Step 1: Obtain the training set;

[0010] The training set includes sentiment sample data. o The sentiment sample data w in the training set o All data are labeled, with the label representing sentiment (s).

[0011] Step 2: Establish a dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation; obtain the trained dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation.

[0012] In step two, a dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation is established. The specific process for obtaining the trained dual sentiment classification model DCA based on diffusion counterfactual data augmentation is as follows:

[0013] The Dual Sentiment Classification (DCA) model based on diffusion counterfactual data augmentation includes: a text generation layer, a dual discrimination layer, a contrastive learning layer, and a dual sentiment classification layer.

[0014] S1: Take the original samples w from the training set o The input is processed by the text generation layer to construct the antonym guidance paradigm w. g ;

[0015] S2: Construct a counterfactual diffusion generative network model to obtain a trained counterfactual diffusion generative network model;

[0016] The antonym-guided paradigm w g and the original sample w o Input a trained counterfactual diffusion generative network model and output a negative sample w. a ;

[0017] S3: Construct the original edge discriminator C o Obtain the trained original edge discriminator C o ;

[0018] Constructing a discriminator for antisense edges C a ; Obtain the trained antisense edge discriminator C a ;

[0019] The dual-discrimination layer uses a trained original edge discriminator C oand the trained antisense edge discriminator C a The trained counterfactual diffusion generative network model is optimized to obtain an optimized counterfactual diffusion generative network model; the counterfactual guidance paradigm w is then applied. g and the original sample w o Input the optimized counterfactual diffusion generative network model, and output the updated counterfactual sample w. a1 ;

[0020] S4: Perform final optimization on the optimized counterfactual diffusion generative network model obtained in S3 to obtain a qualified counterfactual diffusion generative network model. The qualified counterfactual diffusion generative network model is then used as the input for the negation guidance paradigm w. g and the original sample w o Output qualified negative samples w a2 ;

[0021] S5: Use the trained original edge discriminator C obtained in S3. o The original edge classifier and antisense edge classifier are used as the sentiment classification layer; then the original sample w o And the qualified negative samples w generated by S4 a2 Samples are transmitted in the form of pairs to the sentiment classification layer for dual sentiment classification.

[0022] The beneficial effects of this invention are as follows:

[0023] This invention proposes a dual sentiment classification neural network model, DCA, based on diffuse counterfactual data augmentation. This framework combines a counterfactual diffuse generation network model, a discriminator, and a dual sentiment classification model, integrating these components into a framework utilizing reinforcement learning and contrastive learning techniques. The diffuse model has been applied to the field of controlled text generation. Compared to traditional methods such as GANs, the diffuse model often exhibits a more stable training process and can generate diverse content without retraining. Therefore, content control is possible, and content diversity is particularly valuable.

[0024] In the generation phase, multi-label learning is first used to design the optimal antonym paradigm for the original samples. Next, contrastive learning between the antonym paradigm and the original samples guides the sampling process of the diffusion model. By integrating rewards from the discriminator, diverse antonym samples with controllable sentiment labels and coherent semantics are generated. In the sentiment classification phase, a dual sentiment classifier is built, utilizing a pre-trained original sample predictor and an antonym sample predictor from the discriminator. Both predictors are used as sentiment classifiers for dual sentiment classification. By selecting the classifier with higher confidence output, the problem of spurious associations within the data, which leads to poor sentiment classification performance in existing technologies, is addressed. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the DCA model of the present invention. Detailed Implementation

[0026] Specific implementation method one: Combining Figure 1 This invention describes the present invention, including

[0027] I. Establish a dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation; obtain a trained dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation;

[0028] 2. Input the sentiment data to be classified into the trained dual sentiment classification model DCA based on diffusion counterfactual data augmentation to complete the dual sentiment classification.

[0029] Specific Implementation Method Two: The difference between this implementation method and Specific Implementation Method One is that...

[0030] The first step involves establishing a dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation; obtaining a trained dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation; the specific process is as follows:

[0031] Step 1: Obtain the training set;

[0032] The training set includes sentiment sample data. o The sentiment sample data w in the training set o All data are labeled, with the label representing sentiment (s); the types of labels on the sample data differ.

[0033] For example: positive emotional samples w o The movie has a captivating and suspenseful plot; I found it very interesting. (Sample w) o Positive emotions serve as the input to the DCA model and as its output.

[0034] The DCA (Dual Sentiment Classification) model, trained with diffusion counterfactual data augmentation, also utilizes validation and test sets.

[0035] The validation and test sets also include sentiment sample data. o The labels on the sample data in the test set are not visible to the DCA model;

[0036] The validation set is used in our final classification process. Before predicting on the test set, we will use the validation set and the Adam optimizer to fine-tune our model parameters to ensure that our sentiment classification model achieves optimal classification performance. After parameter tuning, we input the test set text into the DCA model for text generation and sentiment classification. The labels on the test set are invisible. We compare the labels predicted by the model with the true labels of the test set text, and find that the model's sentiment classification accuracy is significantly improved compared to the original sentiment classification method.

[0037] Step 2: Establish a dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation; obtain the trained dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation.

[0038] The other steps and parameters are the same as in Specific Implementation Method 1.

[0039] In step two, a dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation is established. The specific process for obtaining the trained dual sentiment classification model DCA based on diffusion counterfactual data augmentation is as follows:

[0040] The Dual Sentiment Classification (DCA) model based on diffusion counterfactual data augmentation includes: a text generation layer, a dual discrimination layer, a contrastive learning layer, and a dual sentiment classification layer.

[0041] S1: Take the original samples w from the training set o The input is processed by the text generation layer to construct the antonym guidance paradigm w. g ;

[0042] S2: Construct a counterfactual diffusion generative network model to obtain a trained counterfactual diffusion generative network model;

[0043] The antonym-guided paradigm w g and the original sample w o Input a trained counterfactual diffusion generative network model and output a negative sample w. a ;

[0044] S3: Construct the original edge discriminator C o Obtain the trained original edge discriminator C o ;

[0045] Constructing a discriminator for antisense edges C a ; Obtain the trained antisense edge discriminator C a ;

[0046] The dual-discrimination layer uses a trained original edge discriminator C o and the trained antisense edge discriminator C aThe trained counterfactual diffusion generative network model is optimized to obtain an optimized counterfactual diffusion generative network model; the antisense guidance paradigm w is then applied. g and the original sample w o Input the optimized counterfactual diffusion generative network model, and output the updated counterfactual sample w. a1 ;

[0047] S4: Perform final optimization on the optimized counterfactual diffusion generative network model obtained in S3 to obtain a qualified counterfactual diffusion generative network model. The qualified counterfactual diffusion generative network model is then used as the input for the negation guidance paradigm w. g and the original sample w o Output qualified negative samples w a2 ;

[0048] S5: Use the trained original edge discriminator C obtained in S3. o The original edge classifier and antisense edge classifier are used as the sentiment classification layer; then the original sample w o And the qualified negative samples w generated by S4 a2 Samples are transmitted to the sentiment classification layer in the form of sample pairs for dual sentiment classification.

[0049] Dual sentiment classification is implemented when the maximum probability of the two sentiment levels in the original edge classifier is higher than the maximum probability of the two sentiment levels in the antisense edge classifier; in this case, the sentiment classification of the original edge classifier will be used. Conversely, since we are predicting the original sample w... o If the sentiment is positive, then we will use the opposite sentiment predicted by the negative edge classifier as the standard, and the negative sample w a The opposite emotion, i.e., the original sample w o The emotions.

[0050] The other steps and parameters are the same as in one of the specific implementation methods one or two.

[0051] Specific Implementation Method Four: The difference between this implementation method and Specific Implementation Methods One to Four is that...

[0052] In S1, the original emotional sample sequence w from the training set will be used. o Input to the text generation layer generates an antonym guide paradigm w g The specific process is as follows:

[0053] S1.1: Take the original sample w o The stop words in the original sample w are set as non-replacement words. o w consists of the original sample sequence o ={w1,w2,...,w nThe original sample sequence consists of n words, where n is a positive integer. Stop words are words unrelated to sentiment to prevent the intrusion of irrelevant information. Stop words are a list of words from the Python NLTK library, containing words such as "the," "and," and "is." They generally contribute little to the meaning of the text and may introduce noise in text processing tasks.

[0054] S1.2: Transfer the original sample w o Adjectives, adverbs, and verbs containing antonyms were replaced with their corresponding antonyms from the English vocabulary database WordNet; the original sample w o The adjectives, adverbs, and verbs in the text can influence the mood of sentences composed of the original sample sequences;

[0055] S1.3: Transfer the original sample w o For adjectives, adverbs, and verbs that do not have antonyms, as well as non-stop words, replace them with corresponding synonyms from the English vocabulary database WordNet;

[0056] S1.4: The original samples after replacement based on S1.1 to S1.3 and the sentiment labels of the original samples (w o ,s) Generates antonymous guiding paradigms and opposite sentiment labels Where s represents the original sample w o Emotional tags, Representing the opposite emotional label.

[0057] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0058] Specific Implementation Method Five: The difference between this implementation method and Specific Implementation Methods One to Four is that...

[0059] For the original sample sequence w o ={w1,w2,...,w n Each word w in} k Supervised multi-label classification is used to calculate the replacement word w for the j-th word in the English vocabulary database WordNet. k The probability of; the specific process is as follows:

[0060]

[0061] in, This indicates that the j-th word in the WordNet English vocabulary database belongs to the original sample w. o The kth word w k The probability of replacing the word set. Represents the original sample w o The kth word w kFor the j-th word in the WordNet English vocabulary database, exp() represents the exponential function, h k For the word w k The hidden representation of W j Let b be the weight matrix. j The deviation is m, where m is the size of the WordNet English vocabulary database.

[0062] Supervised multi-label classification is an algorithm applied to text classification. We design a binary classifier for each word in WordNet, where m is the number of words in WordNet, meaning we have m binary classifiers. Their function is to determine whether a word belongs to the replacement word set. Specifically, we obtain w... o The hidden representation hk of each word in WordNet is compared with each of the m words in WordNet to classify whether the j-th word in WordNet belongs to the replacement word. This method is called a supervised multi-label classification algorithm.

[0063] Then the original sample sequence w o ={w1,w2,...,w n Words with a substitution probability lower than 0.57 and words not included in the WordNet English vocabulary database have their substitution probability set to zero. This process determines whether to substitute word w with the j-th word from the WordNet database. k Normalized probability distribution w k ~Multinomial(P k );

[0064] Determine whether to replace word w with the j-th word from the WordNet English vocabulary database. k Normalized probability distribution w k ~Multinomial(P k The specific process is as follows:

[0065]

[0066] Among them, P k This indicates that for the original sample w o The kth word w k The multinomial word distribution of substitution probabilities in the WordNet English vocabulary database. `normalize()` represents the normalization representation of the probabilities belonging to the substitution set in the WordNet English vocabulary database. This indicates that the first word in the WordNet English vocabulary database belongs to the original sample w. o The kth word w k The probability of the replacement word set, Multinomial, represents the probability of the original sample w. ow1 to w n The polynomial formed by sampling

[0067] Determining a word with a low probability of being replaced involves assessing the likelihood of the j-th word being a replacement word during the multi-label classification process. If this probability is too low, we set it to 0.

[0068] Words not included in the WordNet English vocabulary database have their substitution probability set to 0 because their substitution probability cannot be calculated.

[0069] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0070] Specific Implementation Method Six: The difference between this implementation method and Specific Implementation Methods One to Five is that...

[0071] The specific process of constructing the counterfactual diffusion generative network model in S2 and obtaining the pre-trained counterfactual diffusion generative network model is as follows:

[0072] Construct a counterfactual diffusion generative network model, and use the original sample w o The antithesis guidance paradigm constructed with S1 w g The sample pairs are combined as the training dataset for the counterfactual diffusion generative network model; the antisense guidance paradigm w g and the original sample w o The input is fed into the counterfactual diffusion generative network model for joint training to obtain a pre-trained counterfactual diffusion generative network model.

[0073] The antithesis guiding paradigm w constructed in S1 in S2 g and the original sample w o The specific process of inputting the data into the diffusion model for joint training to obtain a pre-trained counterfactual diffusion generative network model is as follows:

[0074] S2.1: Take the original sample w o The antithesis guidance paradigm constructed with S1. g Serialization is denoted as discrete sample The obtained discrete samples A positive conditional noise addition process is performed by embedding into a continuous feature space to obtain the word vector x0. The specific process is as follows:

[0075]

[0076] in, This represents discrete samples. Perform a forward process of continuous diffusion, where N() represents a multidimensional normal distribution, Emb() represents the embedding function, β0 is the amount of noise added in the time step, and I is the identity matrix;

[0077] Diffusion models can only handle continuous variables. However, text is discrete because it is composed of individual words. Therefore, we emphasize embedding discrete samples into a continuous feature space using an embedding function, which is necessary for diffusion models to process them. Here, discrete samples refer to the concatenated wo+g.

[0078] S2.2: Perform a positive conditional noise addition process T times on the word vector x0 obtained in S2.1, where T is a positive integer; the T-step positive conditional noise addition process yields a series of potential Gaussian noise variables x1,...,x T , x1,...,x T Each Gaussian noise variable in the equation represents an intermediate step output in the forward conditional noise addition process. After the Tth forward conditional noise addition process, the complete Gaussian noise x is obtained. T The specific process is as follows:

[0079] During the positive conditional noise addition process at each time step t∈(1,2...T), the Gaussian noise x obtained in the previous step... t-1 Add β t Gaussian noise, to obtain the Gaussian noise x at time step t. t After performing the Tth positive conditional noise addition process, the complete Gaussian noise x is obtained. T The formula is as follows:

[0080]

[0081] Where q(x) t |x t-1 ) represents Gaussian noise x t-1 The purpose of performing a continuous diffusion forward process divided into T steps is to add a small amount of noise at each step, allowing the network to learn the noise added at each time step to guide the reverse denoising process. This process involves the network learning the noise added at each time step, and the learned parameters are used to guide the reverse denoising process.

[0082] S2.3: Complete the Gaussian noise x obtained in S2.2 T Perform a T-step reverse conditional denoising process to obtain the reverse-denoised word vector x0. The specific process is as follows:

[0083] p θ (x t-1 |x t ,t)=N(x t-1 μ θ (x t ,t),Σ θ (x t ,t))

[0084] Where, p θ (x t-1 |x t ,t) represents Gaussian noise x t Perform a reverse conditional denoising process, μ θ (·) and Σ θ (·) are the Gaussian distribution parameters learned during the inverse conditional denoising process;

[0085] We will use x t-1 In Replace with w o Complete the condition modeling.

[0086] In each diffusion sampling step, we reparameterize x t By applying a rounding operation and projecting it back into the word embedding space, we can infer that the core of our generative process is learning the data distribution between the original samples and the antisense paradigm, rather than controlling them through a large number of classifiers.

[0087] Unlike the traditional diffusion model described above, our reverse diffusion denoising process involves... t-1 China belongs to Partial use of w o Replacement (which can be understood as because) If a series of elements are embedded into a continuous space, then x t-1 Actually it is and (composed of embedded components) So in the inverse denoising process, we keep performing the above replacements, which is equivalent to the inverse denoising only acting on the Therefore, we call it x. t It is a variable that is partially Gaussian noise.

[0088] Combination Figure 1 Looking at the lower left, all the noise points are drawn in the upper half. Therefore, this invention denoises x0 from xt, which is partially Gaussian noise. This x0 still consists of two parts, where the x0 part remains unchanged because it is constantly being replaced. The final product is w. g Partially. The word vector x0 after reverse denoising is not strictly equal to the initial x0, but only close to the original x0.

[0089] S2.4: Calculate the variational lower bound based on the forward noise addition process in S2.2 and the reverse noise reduction process in S2.3: Use the variational lower bound as the loss function to train the counterfactual diffusion generative network model, obtaining a pre-trained counterfactual diffusion generative network model; the specific process is as follows:

[0090]

[0091] Among them, Eq (·) indicates expectation, D KL (q(x T |x0)||p θ (x T )) indicates the forward diffusion process. This represents the reverse process of diffusion. This indicates that discrete samples are embedded in a continuous space. For the rounding operation, D KL The formula for calculating the KL divergence between Gaussian distributions is given by x. T This represents Gaussian noise.

[0092] The variational lower bound, as a loss function, refers to the mean square error between the actual noise and the model's predicted noise.

[0093] The reason for using variational lower bound is that the original training loss of the diffusion model is not convenient to optimize. After deriving and decomposing the original loss, it is transformed into a variational lower bound loss function to replace the original loss function.

[0094] Other steps and parameters are the same as in any of the specific implementation methods one to five.

[0095] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One to Six in that, in step S3, the original edge discriminator is constructed to obtain the trained original edge discriminator; the specific process is as follows:

[0096] When the original edge discriminator is an LSTM model, the original sample w o The corresponding sentiment labels s are used as the training set for the original edge discriminator; each original sample w is used as the training set for the edge discriminator. o The edge discriminator is transformed into a word embedding sequence by an embedding function, and then trained using supervised learning to obtain a well-trained edge discriminator.

[0097] When the original edge discriminator is a pre-trained model Bert-base or Bert-large, the original sample w o The original edge discriminator C is trained using its corresponding sentiment tags s as the training set; the original edge discriminator C is trained using a fine-tuning method. o This yields a well-trained original edge discriminator. Fine-tuning methods are divided into Fine-tune and prompt-tune. Fine-tune, also known as full-parameter fine-tuning, is the method that BERT has always used for fine-tuning models. All parameter weights participate in the update to adapt to the domain data, and it has good results.

[0098] In step S3, a sniffing edge discriminator is constructed; the trained sniffing edge discriminator is obtained; the specific process is as follows:

[0099] When the antisense edge discriminator is an LSTM model, the antisense samples w generated by the pre-trained counterfactual diffusion generative network model obtained in S2 are used. a and its corresponding emotional tags As a training set, a supervised learning method is used to train the antisense edge discriminator to obtain a well-trained antisense edge discriminator.

[0100] Each negative sample w a By transforming the text into a word embedding sequence through an embedding function, and through supervised learning, the discriminator can extract sentiment information from the text in order to ultimately predict the associated sentiment labels.

[0101] When the antisense edge discriminator is a pre-trained model Bert-base or Bert-large, the antisense samples w generated by the pre-trained counterfactual diffusion generation network model obtained in S2 will be used. a The corresponding sentiment tags s are used as the training set; the antisense edge discriminator C is trained using a fine-tuning method. a Obtain the trained antisense edge discriminator C a Fine-tuning methods are divided into two types: fine-tuning and prompt-tuning. Fine-tuning, also known as full-parameter fine-tuning, is the method that BERT models have always used for fine-tuning. All parameter weights participate in the update to adapt to the domain data, and it has good results.

[0102] Provide the model with the original sample w a and its corresponding emotional tags Through the fine-tuning process, w a Convert to an input format suitable for the BERT model. During fine-tuning, we will convert each original sample w... a The corresponding sentiment label s is fed into the discriminator to adjust the parameters of the BERT model, making it better suited for sentiment classification tasks. The antisense edge discriminator C is trained using the above method. a The other steps and parameters are the same as in any of the specific implementation methods one through six.

[0103] Specific Implementation Method Eight: The difference between this implementation method and Specific Implementation Methods One to Seven is that...

[0104] The dual-discrimination layer in S3 uses the trained original edge discriminator and the trained antisense edge discriminator to optimize the trained counterfactual diffusion generation network model, and the specific process of obtaining the optimized counterfactual diffusion generation network model is as follows:

[0105] S3.1: Transfer the negative sample w a Input into the trained original edge discriminator C o Original edge discriminator C oThe inner layer contains multiple hidden layers and the output layer of the original edge discriminator. Each layer consists of weights and activation functions. The output of each hidden layer serves as the input to the next hidden layer, and the output h of the final hidden layer is... o As input to the output layer of the original edge discriminator, h o Represents the antonym sample w a In the original edge discriminator C o Hidden representation within.

[0106] The antonymous sample w a Input to the anti-edge discriminator C a Antisense edge discriminator C a The innermost layer contains multiple hidden layers, each consisting of weights and an activation function; the output of each hidden layer serves as the input to the next hidden layer, and the final hidden layer outputs h. a As the input to the output layer of the antisense edge discriminator, h a Represents the antonym sample w a In the original edge discriminator C a Hidden representation within;

[0107] The obtained h o and h a Perform dual-sentiment prediction;

[0108] They obtain hidden representations, denoted as ho and h, through a network of original edge discriminators and antisense edge discriminators, respectively. a This approach enables dual sentiment prediction. When using LSTM as the discriminator network model, it captures temporal information in the text, converting it into a sequence of word embeddings through an embedding function. Utilizing the recurrent structure of LSTM, it understands the word order relationships within sentences, capturing the complexity of sentiment expression. When using Bert-base or Bert-large as the discriminator network model, based on the Transformer architecture and pre-trained on large-scale corpora, it deeply understands the context and provides powerful modeling capabilities for sentence-level sentiment analysis. Fine-tuning is then performed on pre-trained Bert-base or Bert-large models.

[0109] The output layers of both the original edge discriminator and the negated edge discriminator are softmax layers, with the following output:

[0110] p ori (s|w a = softmax(W) ori h o +b ori )

[0111] p ant (s|wa = softmax(W) ant h a +b ant )

[0112] Among them, W ori and b ori C represents o The parameter W ant and b ant C represents a Parameters; C a The parameters change with the antisense sample w a The generation is dynamically updated. We prevent the original predictor C from being generated by preventing gradient backpropagation. o Parameter updates;

[0113] W ori and b ori These are the parameters of the original edge discriminator, which are trained using the original sample set. Therefore, after training, these two parameters are no longer updated, meaning they no longer change with the generation of negative samples. Preventing gradient backpropagation means that the original edge discriminator is initially trained only on the original sample set Do, and later, although it participates in the generation of negative samples w... a It identifies, but does not perform negative sample identification. a The backward gradient is propagated to prevent the gradient from backpropagating, thus stopping the original edge discriminator C. o The network parameters are updated. This step involves updating the antisense sample w. a This is reflected in the process of performing dual sentiment discrimination. The parameters of the antisense edge discriminator are obtained through the antisense sample w. a Trained, so it won't prevent it in C a Gradient backpropagation.

[0114] The parameter W of the anti-edge discriminator ant and b ant Because initially there were no antisense samples; the antisense samples were generated later. The generated antisense samples w a This is used to train a discriminator for an antisense edge. This training process keeps Want and Bant updated, so it is described here as a dynamic update.

[0115] S3.2: Dual discrimination layer based on the original edge discriminator C o and antisense edge discriminator C a The output is double-authenticated, and a reward is generated simultaneously. The specific process is as follows:

[0116]

[0117] Where α and γ are negatively correlated tradeoff parameters that sum to 1, in Ca During the cold start phase, with C a Performance improvement occurs as α gradually increases from 0 to 1; the cold start phase means that initially, our antisense edge discriminator is not trained with generated antisense samples, and the antisense sample w... a It was generated later; it was generated one negative sample at a time. a It is gradually trained. As the number of generated negative samples increases, the negative edge discriminator begins to train gradually, and this process is called cold start.

[0118] S3.3: Using a gradient-based strategy (similar to random sampling), M negative samples are drawn for dual discrimination as described in S3.2. The resulting M rewards are calculated, and the average reward of these M rewards is used as the baseline reward r. avg and based on the obtained baseline reward r avg Reinforcement learning is performed (the extraction step is carried out after the counterfactual diffusion generative network generates the sample w). o Antonyms generated through counterfactual diffusion w a It's not unique; multiple antonyms were generated. We will use the average reward from the calculated antonyms as a supervision signal for the generated antonym samples.

[0119] The specific process is as follows:

[0120] When r(w) a (Above average reward r) avg At the same time, adding a counterfactual diffusion generative network model generates countersense samples w a The probability when r(w) a Less than or equal to the average reward r avg At this time, the negative sample w is reduced. a Rewards and baseline rewards r avg The difference is the generated antisense sample w. a If the actual reward is negative or 0, then the counterfactual diffusion generator network will reduce the number of such countersense samples w generated. a The generation,

[0121] The formula for the loss function in reinforcement learning is as follows:

[0122] L r =-log(r(w) a )-r avg )P G (w a |w o )

[0123] Where r(w) a )-r avg Represents the antonym sample w aRewards and baseline rewards r avg The difference is used as the negative sample w a The actual calculated reward, P G (w a |w o ) represents a counterfactual diffusion generative network model given the original sample w. o Generate negative samples w a The probability of.

[0124] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0125] Specific Implementation Method Nine: The difference between this implementation method and Specific Implementation Methods One through Eight is that...

[0126] In step S4, the optimized counterfactual diffusion generator network model obtained in step S3 is finally optimized to obtain a qualified counterfactual diffusion generator network model; the specific process is as follows:

[0127] The original sample w o The antithesis guidance paradigm obtained by S1 w g and the antisense sample w generated by S2 a Mapped to different dimensions of the contrastive learning layer, the contrastive learning layer learns contrastive representations using an embedding function f(·); the contrastive loss is used to finally optimize the optimized counterfactual diffusion generative network model obtained in S3; the specific process is as follows:

[0128] In S4, the original sample w o Antonym Guiding Paradigm w g and the generated negative sample w a Mapped to different dimensions of the contrastive learning layer, the contrastive learning layer uses an embedding function f(·) to learn contrastive representations; the specific process is as follows:

[0129] S4.1: The antonymous guiding paradigm w obtained in S1 g As an anchor point, the updated antisense samples w generated by the S3-optimized counterfactual diffusion generative network model are used. a1 As a positive sample, the original sample w o As negative samples, they are used for contrastive learning. The loss function formula for contrastive learning is as follows:

[0130]

[0131] Where ε represents the safety margin, and f(·) represents the embedding function. Let represent the square of the L2 norm, and max(·) denote the maximum value function.

[0132] S4.2: Obtain L from S4.1 c L obtained from S3.3r L obtained from S2.4 vlb The sum of these losses forms the overall loss function used to train and optimize the counterfactual diffusion generative network model, resulting in a qualified counterfactual diffusion generative network model. The overall loss function formula is as follows:

[0133] L = L vlb +L r +L c

[0134] Among them, L vlb To generate the loss function of the diffusion network, L r To reinforce the learning loss function, L c To compare the learning loss function.

[0135] Based on the original samples input to the contrastive learning layer and their associated sentiments (w) o ,s) and the antonymous guiding paradigm and its associated emotions We will spread the generation network loss L vlb Its function is to enable the generator network to generate outputs that are similar to the input; reinforcement learning loss L r Its role is to jointly optimize the counterfactual diffusion generative network model and the discriminator; and the contrastive learning loss L. c Its function is to bring the antisense samples w closer together. a With the antonym-guided paradigm w g The distance, increasing the distance of the original sample w o Antonym Guiding Paradigm w g The distance between the three is used as the total loss, which is fed back to the counterfactual diffusion generative network model to help it generate qualified countersense samples and their correct associated sentiments.

[0136] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.

[0137] Specific Implementation Method Ten: The difference between this implementation method and Specific Implementation Methods One through Nine is that...

[0138] In S5, the trained original edge discriminator C obtained in S3 is... o The original edge classifier and antisense edge classifier are used as the sentiment classification layer; then the original sample w o And S4 generates qualified negative samples w a2 The samples are transmitted in the form of pairs to the sentiment classification layer for dual sentiment classification. The specific process is as follows:

[0139] Calculate the confidence scores of the original edge classifier and the antisense edge classifier. When the confidence score of the original edge classifier exceeds the confidence threshold μ of the antisense edge classifier, the original sample w... oThe sentiment label adheres to the results of the original classifier, and conversely, the original sample w o The sentiment label is considered a qualified antonym sample. a2 Emotions; Obtaining the original sample w o The formula for determining the sentiment label is as follows:

[0140]

[0141] Where μ represents the confidence threshold, p(s|w o ) represents the original sample w o Sentiment prediction, p ori (s|w o ) represents the original edge classifier for the original sample w o Sentiment prediction, p ant (s|w a2 ) indicates that the antisense edge classifier has a good understanding of qualified antisense samples w. a2 Sentiment prediction.

[0142] The specific process is explained as follows: Original sample w o and the antisense sample w a There is a one-to-one correspondence. We will... o Transmitted to the original edge classifier, w o The corresponding w a The data is then fed to the antisense edge classifier for sentiment classification. The confidence threshold is a value we set ourselves, ranging from [0,1]. It's important to emphasize that the predictor's return is a pair of numerical probabilities. The original edge classifier and the antisense edge classifier classify the original sample w respectively. o and the antisense sample w a The sentiment prediction form is (negative probability, positive probability), where the sum of the two probabilities is 1. Taking (a, b) as an example, the sum of the two values ​​a and b is 1, where a represents the probability of negative sentiment and b represents the probability of positive sentiment. Then, by comparing the original side classifier with w... o The sentiment probability distribution and the antisense edge classifier for w a The sentiment probability distribution can be used to obtain a high-confidence sentiment. If the sentiment comes from the original side classifier, then the sentiment predicted by the original side classifier is taken as the standard; if the sentiment comes from the antisense side classifier, then the sentiment predicted by the antisense side classifier is taken as the opposite sentiment.

[0143] To make it easier to understand, let's take an example w. o →C o w a →C a If C o The given prediction is (0.2, 0.8), C aThe given prediction is (0.3, 0.7), so let's take C as the value. o Based on the predicted sentiment, it is positive. Conversely, if this C... a If the predicted probability is higher, then the sentiment of the original sample sentence is negative. What we want is the original sample w. o The confidence threshold μ is a user-defined value. For binary classification tasks, it is set to 0.8 / 0.52 for LSTM / BERT models; for five-class classification tasks, it is set to 0.4 / 0.22 for LSTM / BERT models. The purpose of setting the confidence threshold is to control the probability distribution of sentiment within a suitable range based on the results of a classifier. Other steps and parameters are the same as in one of the specific implementation methods one to nine.

[0144] The above description is merely of preferred embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention, and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A dual sentiment classification method based on diffusion counterfactual data enhancement, characterized in that, Includes the following steps: I. Establish a dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation; obtain a trained dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation; The specific process is as follows: The Dual Sentiment Classification (DCA) model based on diffusion counterfactual data augmentation includes: a text generation layer, a dual discrimination layer, a contrastive learning layer, and a dual sentiment classification layer. S1: Obtain the original samples from the training set. The input is processed in the text generation layer to construct the antonym guidance paradigm. ; S2: Construct a counterfactual diffusion generative network model to obtain a trained counterfactual diffusion generative network model; Antonym Guiding Paradigm and original sample Input a trained counterfactual diffusion generative network model and output negative samples. ; S3: Construct the original edge discriminator Obtain the trained original edge discriminator ; Constructing a negative edge discriminator ; Obtain a trained antisense edge discriminator ; The dual-discrimination layer uses a trained original edge discriminator. and trained antisense edge discriminator The trained counterfactual diffusion generative network model is optimized to obtain an optimized counterfactual diffusion generative network model; the counterfactual guidance paradigm is then applied. and original sample Input the optimized counterfactual diffusion generative network model and output updated counterfactual samples. ; S4: Perform final optimization on the optimized counterfactual diffusion generative network model obtained in S3 to obtain a qualified counterfactual diffusion generative network model. The qualified counterfactual diffusion generative network model is then used as the input for the negation guidance paradigm. and original sample Output qualified negative samples The specific process is as follows: S4.1: Obtain the antonymous guiding paradigm from S1 As anchor points, the updated antisense samples generated by the S3-optimized counterfactual diffusion generative network model are used. As positive samples, the original samples As negative samples, they are used for contrastive learning. The loss function formula for contrastive learning is as follows: in Indicates the safety margin. Indicates an embedded function. express The square of the norm, Represents the maximum value function. S4.2: The contrastive learning loss function obtained in S4.1 The reinforcement learning loss function used in S3 to optimize the training of the counterfactual diffusion generative network model. The loss function of training a counterfactual diffusion generative network model with S2 The sum of these losses forms the overall loss function used to train and optimize the counterfactual diffusion generative network model, resulting in a qualified counterfactual diffusion generative network model. The overall loss function formula is as follows: in, To train the loss function of the counterfactual diffusion generative network model, To reinforce the learning loss function, To learn the loss function by comparison; S5: The original edge discriminator trained in S3 The original edge classifier and antisense edge classifier are used as the sentiment classification layer; then the original samples are... Qualified negative samples generated by S4 Samples are transmitted in pairs to the sentiment classification layer for dual sentiment classification.

2. Input the sentiment data to be classified into the trained dual sentiment classification model DCA based on diffusion counterfactual data augmentation to complete the dual sentiment classification.

2. The dual sentiment classification method based on diffusion counterfactual data enhancement according to claim 1, characterized in that, In step one, a dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation is established; and a trained dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation is obtained. The specific process is as follows: Step 1: Obtain the training set; The training set includes sentiment sample data. The sentiment sample data in the training set All data are labeled, with the label representing sentiment (s). Step 2: Establish a dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation; obtain the trained dual sentiment classification model (DCA) based on diffusion counterfactual data augmentation.

3. The dual sentiment classification method based on diffusion counterfactual data enhancement according to claim 2, characterized in that, S1 will contain the original emotional sample sequence from the training set. Input to the text generation layer generates a negation guidance paradigm The specific process is as follows: S1.1: Take the original sample The stop words in the original sample are set as non-replacement words. Composed of the original sample sequence The original sample sequence consists of n words; n is a positive integer; S1.2: Take the original sample For adjectives, adverbs, and verbs containing antonyms, replace them with the corresponding antonyms from the English vocabulary database WordNet; S1.3: Take the original sample For adjectives, adverbs, and verbs that do not have antonyms, as well as non-stop words, replace them with corresponding synonyms from the English vocabulary database WordNet; S1.4: The original samples after replacement based on S1.1 to S1.3, and the sentiment labels of the original samples. Generate antonymous guiding paradigms and opposite sentiment labels. ,in Represents the original sample Emotional tags, Representing the opposite emotional label.

4. The dual sentiment classification method based on diffusion counterfactual data enhancement according to claim 3, characterized in that, In S1.2, the original sample Adjectives, adverbs, and verbs containing antonyms are replaced with their corresponding antonyms from the WordNet English vocabulary database; the specific process is as follows: For the original sample sequence Each word in Supervised multi-label classification is used to calculate the replacement word for the j-th word in the English vocabulary database WordNet. The probability of; the specific process is as follows: in, This refers to the first word in the WordNet English vocabulary database. Each word belongs to the original sample. The Middle Word The probability of replacing the word set. Represents the original sample The Middle Word The corresponding English vocabulary database WordNet, number The word exp() represents the exponential function. For words The hidden representation, This is the weight matrix. For deviation, This refers to the size of the English vocabulary database WordNet; Then the original sample sequence Words with a substitution probability lower than 0.57 and words not included in the WordNet English vocabulary database have their substitution probability set to zero. This process is used to determine whether to substitute the j-th word from the WordNet database. Normalized probability distribution ~ ; Determine whether to use the j-th word from the WordNet English vocabulary database as a replacement word. Normalized probability distribution ~ The specific process is as follows: in, This indicates that for the original sample The Middle Word Multinomial word distribution of replacement probabilities in the English vocabulary database WordNet ( ) represents the normalized representation of the probability of belonging to the replacement word set in the WordNet English vocabulary database. This indicates that the first word in the WordNet English vocabulary database belongs to the original sample. The Middle Word The probability of the replacement word set, Indicates the original sample In arrive The polynomial formed by sampling.

5. The dual sentiment classification method based on diffusion counterfactual data enhancement according to claim 4, characterized in that... The specific process of constructing the counterfactual diffusion generative network model in S2 and obtaining the pre-trained counterfactual diffusion generative network model is as follows: S2.1: Take the original sample The antithesis guidance paradigm constructed with S1 Serialization is denoted as discrete sample The obtained discrete samples Word vectors are obtained by performing a positive conditional noise addition process within a continuous feature space. The specific process is as follows: in, This represents discrete samples. Embedding into a continuous feature space to perform a continuous diffusion forward process. () represents a multidimensional normal distribution, and Emb() represents the embedding function. The amount of noise added during the time step. It is the identity matrix; S2.2: Convert the word vectors obtained in S2.1 Perform a T-step positive conditional noise addition process to obtain T Gaussian noise variables. , Each Gaussian noise variable in the equation represents an intermediate step output in the forward conditional noise addition process, where T is a positive integer; the specific process is as follows: At each time step During the positive conditional noise addition process, the Gaussian noise obtained in the previous step... Add Gaussian noise, to obtain the Gaussian noise at time step t. The formula is as follows: in, Indicates Gaussian noise A continuous diffusion process is carried out in the forward direction; S2.3: Gaussian noise obtained in S2.2 Perform a T-step inverse conditional denoising process to obtain the inversely denoised word vectors. The specific process is as follows: in, Indicates Gaussian noise Perform a reverse conditional denoising process. and These are the Gaussian distribution parameters learned during the inverse conditional denoising process; S2.4: Calculate the variational lower bound based on the forward noise addition process in S2.2 and the reverse noise reduction process in S2.3; use the calculated variational lower bound as the loss function for training the counterfactual diffusion generative network model to obtain the pre-trained counterfactual diffusion generative network model; The formula for the loss function of the training counterfactual diffusion generative network model is as follows: in, Represents the expectation function, This indicates the forward diffusion process. This represents the reverse process of diffusion. This indicates that discrete samples are embedded in a continuous space. For rounding operations, This represents the formula for calculating the KL divergence between Gaussian distributions. This represents Gaussian noise.

6. The dual sentiment classification method based on diffusion counterfactual data enhancement according to claim 5, characterized in that, The original edge discriminator is constructed in S3. Obtain the trained original edge discriminator The specific process is as follows: When the original edge discriminator When using an LSTM model, the original samples and its corresponding emotional tags As the original edge discriminator The training set; to obtain the trained original edge discriminator. ; When the original edge discriminator When pre-training the Bert-base or Bert-large model, the original samples and its corresponding emotional tags As the original edge discriminator The training set; to obtain the trained original edge discriminator. ; The antisense edge discriminator is constructed in S3. ; Obtain a trained antisense edge discriminator The specific process is as follows: When the antisense edge discriminator When using an LSTM model, the antisense samples generated by the pre-trained counterfactual diffusion generative network model obtained in S2 are used. and its corresponding emotional tags As a training set, a well-trained antisense edge discriminator is obtained. ; When the antisense edge discriminator When pre-training the Bert-base or Bert-large model, the antisense samples generated by the pre-trained counterfactual diffusion generative network model obtained in S2 are used. and its corresponding emotional tags Used as a training set; to obtain a trained anti-edge discriminator. .

7. The dual sentiment classification method based on diffusion counterfactual data enhancement according to claim 6, characterized in that, The dual discrimination layer in S3 uses a trained original edge discriminator. and trained antisense edge discriminator The trained counterfactual diffusion generative network model is optimized to obtain an optimized counterfactual diffusion generative network model. The specific process is as follows: S3.1: Redirecting the negative samples Input into the trained raw edge discriminator , get output , will the antonyms Input into the trained antisense edge discriminator , get output ; The original edge discriminator The inner layer contains a hidden layer and an output layer, and the output of the hidden layer... As the original edge discriminator The input of the output layer, where Indicates the negative sample In the original edge discriminator Hidden representation within; The antisense edge discriminator It contains a hidden layer and an output layer, with the hidden layer outputting the output layer. As a negative edge discriminator The input of the output layer, where Indicates the negative sample In the original edge discriminator Hidden representation within; Original edge discriminator and antisense edge discriminator The output layers are all softmax layers, and the output is as follows: in, and express The parameters, and express Parameters; S3.2: Dual discrimination layer based on the original edge discriminator and antisense edge discriminator The output undergoes dual authentication, and a reward is generated simultaneously. The specific process is as follows: in, and It is a negative correlation trade-off parameter that sums to 1. S3.3: Extract the output of the pre-trained counterfactual diffusion generative network model from S2. One antisense sample Perform dual authentication in S3.2 to obtain M rewards, and calculate the average reward of the M rewards as the baseline reward. and based on the baseline reward obtained. Reinforcement learning is performed to obtain an optimized counterfactual diffusion generative network model; the specific process is as follows: when Greater than average reward At the same time, adding a counterfactual diffusion generative network model generates countersense samples. The probability, when Less than or equal to average reward At the same time, reduce the generation of antisense samples by the counterfactual diffusion generative network model. The probability of; The formula for the loss function in reinforcement learning is as follows: in Indicates the negative sample Rewards and baseline rewards The difference as an antonym sample The actual calculated reward, This indicates that the counterfactual diffusion generative network model is given the original samples. Generate negative samples The probability of.

8. The dual sentiment classification method based on diffusion counterfactual data enhancement according to claim 7, characterized in that, In S5, the trained original edge discriminator obtained in S3 will be used. The original edge classifier and antisense edge classifier are used as the sentiment classification layer; then the original samples are... S4 generates qualified negative samples The samples are transmitted in the form of pairs to the sentiment classification layer for dual sentiment classification. The specific process is as follows: Calculate the confidence scores of the original edge classifier and the antisense edge classifier. When the confidence score of the original edge classifier exceeds the confidence score threshold of the antisense edge classifier... At that time, the original sample The sentiment labels adhere to the results of the original classifier, and vice versa. The sentiment label is considered a qualified antonym sample. Emotions; Obtaining raw samples The formula for determining the sentiment label is as follows: in, Indicates the confidence threshold. Represents the original sample Sentiment prediction This indicates the original edge classifier's performance on the original samples. Sentiment prediction This indicates that the negative edge classifier has a good understanding of qualified negative samples. Sentiment prediction.

Images