Emotion analysis method based on low-resource cross-domain diffusion anti-fact data enhancement

Abstract

The invention discloses a sentiment analysis method based on low-resource cross-domain diffusion anti-fact data enhancement, is applied to the technical fields of artificial intelligence, natural language processing and text generation, and relates to a text sentiment classification method. The invention aims to solve the problem that an existing sentiment analysis method cannot well process out-of-domain data due to intra-domain data training of a model. The method comprises the following steps: establishing a CDA2 model; the original target domain sample generation layer is used for generating original target domain guide samples from samples in a training set with emotion tags in a source domain and samples in a test set without emotion tags in a target domain; the anti-fact target domain sample generation layer is used for inputting a training set with an emotion label of a source domain and a generated original target domain guide sample set into a diffusion model in a sample pair form, and the diffusion model generates a qualified anti-fact target domain sample set; and the target domain sample data classification layer performs sentiment classification on the test set of the unmarked sentiment label in the target domain.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence, natural language processing and text generation technology, and in particular to a text sentiment classification method. Background Art

[0002] Sentiment analysis is an important task in Natural Language Processing (NLP), and its core is to extract the implicit emotions in text data. In recent years, its popularity has surged due to its wide range of real-world applications. In recent years, deep learning techniques have made significant progress and achieved remarkable success in sentiment analysis. However, when running under low-resource conditions or encountering data distribution shift between the training domain and the target domain, the performance of traditional sentiment analysis methods that rely on labeled data to train models in the target domain degrades significantly. In order to reduce the dependence on labeled data, cross-domain sentiment analysis has attracted the attention of researchers. Many previous works have resorted to unsupervised domain adaptation techniques, which aim to transfer knowledge from a resource-rich source domain to a target domain with unlabeled data. In the cross-domain sentiment analysis task, most of the existing domain adaptation methods adopt adversarial training to prevent the model from distinguishing samples from a specific domain, thereby transferring knowledge from the source domain to the target domain and trying to learn domain-specific knowledge. Although these methods have achieved promising results, their models are only trained on in-domain labeled data in the source domain, which limits their ability to handle out-of-domain (OOD) data. To address the above limitations, researchers have attempted to design cross-domain data augmentation methods. The key goal is to generate a large number of labeled target domain samples based on labeled source domain samples, thereby achieving knowledge transfer. Research within this framework mainly includes two methods: Masked Language Models (MLM) and Sequence-to-Sequence (Seq2Seq) models. The core idea of ​​both methods is to mask domain-specific terms in source domain samples, and then use the trained MLM or Seq2Seq model to fill in the masked terms to achieve word replacement in the target domain. Although word replacement-based data augmentation methods have shown advantages over feature adaptation methods, they still have some defects: (i) The expanded samples generated by word replacement may cause semantic disruption. (ii) They retain the fixed syntactic structure of the source domain, establishing false associations between the generated text and the source domain. (iii) The diversity of the generated samples is limited by the source domain template. Summary of the invention

[0003] The purpose of the present invention is to solve the problem that current sentiment analysis methods cannot handle out-of-domain (OOD) data well because the model itself is trained on in-domain (IID) data, and to propose a sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement.

[0004] The specific process of sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement is as follows:

[0005] Step 1: Obtain sample data in the sentiment dataset;

[0006] The sample data in the sentiment dataset are all sentiment-labeled data;

[0007] The sample data in the sentiment dataset carry different types of sentiment labels;

[0008] The sample data in the sentiment dataset is divided into a training set and a test set;

[0009] The training set and test set come from different fields;

[0010] The test set sentiment labels are invisible to the model;

[0011] The training set and test set include samples w and sentiment s;

[0012] Step 2: Create CDA 2 Model;

[0013] CDA 2 The model includes: original target domain sample generation layer, counterfactual target domain sample generation layer, and target domain sample data classification layer;

[0014] The original target domain sample generation layer transforms the sample w in the training set with emotion labels in the source domain into s and the unlabeled sentiment labels of the target domain in the test set w t Generate original target domain bootstrap samples w r ;

[0015] The counterfactual target domain sample generation layer is used to convert the training set D with emotion labels in the source domain s And the generated original target domain guided sample set D r The diffusion model is input in the form of sample pairs, and the diffusion model generates a qualified counterfactual target domain sample set D C ;

[0016] A test set D of unlabeled sentiment labels of the target domain t The unlabeled sentiment labels of the target domain in the test set are samples w t composition;

[0017] The counterfactual target domain sample set D C From the counterfactual target domain sample set, sample w c composition;

[0018] The original target domain guide sample set D r The sample w in the sample set is guided by the original target domain r composition;

[0019] The target domain sample data classification layer is used to train the Bert classification model using samples labeled with emotion and domain respectively, and the trained Bert classification model is used to classify the test set D of the target domain without labeled emotion labels. t Perform sentiment classification.

[0020] The beneficial effects of the present invention are:

[0021] This paper proposes a cross-domain adaptive framework CDA for low-resource sentiment analysis 2 , the framework uses counterfactual diffusion enhancement. CDA 2 It aims to alleviate the semantic interruptions and spurious associations caused by fixed syntactic structures in the source domain. The method of the present invention performs well in generating counterfactual target domain samples with outstanding diversity and fluency, which is different from previous studies. The framework constructs a simple word replacement method by implementing conditional guidance of the diffusion generator. Specifically, high-quality original target samples are provided to the generator through domain destruction and domain reconstruction. Next, in the training phase of the diffusion model, a learnable soft absorption state is designed by introducing additional discrete noise in the continuous diffusion process to better adapt to the inherent discrete nature of the text. In addition, the maximum mean deviation (MMD) loss is introduced to supervise the generation process using real target domain unlabeled samples, thereby promoting better data distribution transfer. In the sampling phase, an advanced ordinary differential equation (ODE) accelerator is used to speed up sampling while minimizing the loss of sample quality, thereby generating high-quality counterfactual target samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0023] Specific implementation method 1: The specific process of this implementation method of sentiment analysis based on low-resource cross-domain diffusion counterfactual data enhancement is as follows:

[0024] Step 1: Obtain sample data in the sentiment dataset;

[0025] The sample data in the sentiment dataset are all sentiment-labeled data;

[0026] The sample data in the sentiment dataset has different types of sentiment labels (type of sentiment, positive or negative);

[0027] The sample data in the sentiment dataset is divided into a training set and a test set;

[0028] The training set and test set come from different domains (category of domain, kitchen or book);

[0029] The test set sentiment labels are invisible to the model;

[0030] The training set and test set include samples w and sentiment s;

[0031] For example: Sample w with emotion label from source domain s “The journey of this book fills the reader with joy and anticipation” and text related to kitchen appliances from the target domain without sentiment labels t As CDA 2 The input of the model is the generated text w with the distribution characteristics of the target domain text data c CDA 2 The output of the model;

[0032] Step 2: Create CDA 2 Model;

[0033] CDA 2 The model includes: original target domain sample generation layer, counterfactual target domain sample generation layer, and target domain sample data classification layer;

[0034] The original target domain sample generation layer transforms the sample w in the training set with emotion labels in the source domain into s and the unlabeled sentiment labels of the target domain in the test set w t Generate original target domain bootstrap samples w r ;

[0035] The counterfactual target domain sample generation layer is used to convert the training set D with emotion labels in the source domain s And the generated original target domain guided sample set D r The diffusion model is input in the form of sample pairs, and the diffusion model generates a qualified counterfactual target domain sample set D C ;

[0036] A test set D of unlabeled sentiment labels of the target domain t The unlabeled sentiment labels of the target domain in the test set are samples w t composition;

[0037] The counterfactual target domain sample set D CFrom the counterfactual target domain sample set, sample w c composition;

[0038] The original target domain guide sample set D r The sample w in the sample set is guided by the original target domain r composition;

[0039] The target domain sample data classification layer is used to train the Bert classification model using samples labeled with emotion and domain respectively, and the trained Bert classification model is used to classify the test set D of the target domain without labeled emotion labels. t Perform sentiment classification.

[0040] Further improve the performance of generated counterfactual target domain samples in terms of sentiment and domain;

[0041] Problem definition:

[0042] This paper mainly studies cross-domain sentiment classification in low-resource scenarios. Two domains are considered: source domain and target domain. The source domain set D s Contains labeled data and unlabeled data, and the amount of labeled data is much larger than the amount of unlabeled data. t Including a set of unlabeled data, the goal of cross-domain sentiment classification is to use D s and D t Predict labels for test samples in the target domain.

[0043] The CDA 2 for CounterfactualDiffusionAugmentationforCross-DomainAdaptationin Low-ResourceSentimentAnalysis.

[0044] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that the original target domain sample generation layer transforms the samples w in the training set with emotion labels in the source domain through domain destruction and domain reconstruction under the mask mechanism. s and the unlabeled sentiment labels of the target domain in the test set w t Generate original target domain bootstrap samples w r ;

[0045] The specific process is:

[0046] Step 1. Domain destruction: training set D with sentiment labels in the source domain s The specific domain-related terms in the source domain are masked to obtain the training set D with emotion labels in the masked source domain. s ; The specific process is:

[0047] 1) Training set D s It includes sentence samples, and sentences include words;

[0048] Let w = {w1,w1,…,wj,…,wn} represent the training set D with emotion labels in the source domain s A sample of sentences of length n in , w j Represents a word in a sentence sample;

[0049] 2) Training set D with emotion labels in the source domain s Mask all uni-grams in ;

[0050] 3) Training set D with emotion labels in the source domain s Mask all the uni-grams in ;

[0051] 4) Training set D with emotion labels in the source domain s All uni-grams in the source domain are masked to obtain the training set D with sentiment labels after masking. s ;

[0052] Step 1 and 2: Domain Reconstruction: Samples w in the test set with unlabeled sentiment labels in the target domain t and the training set D with emotion labels in the source domain after masking s , get the original target domain guide sample w r .

[0053] The other steps and parameters are the same as those in the first embodiment.

[0054] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that the training set D with emotion labels in the source domain in 2) is s Mask all uni-grams in ; the specific process is:

[0055] Calculate the masking score M(w j ,D s ,D t ), if the masking score of a unigram (a word in w) is greater than the threshold t, the unigram is masked; if the masking score of a unigram is less than or equal to the threshold t, the unigram is not masked.

[0056] The other steps and parameters are the same as those in the first embodiment.

[0057] Specific implementation method 4: This implementation method is different from one of the specific implementation methods 1 to 3 in that the training set D with emotion labels in the source domain in 3) iss All the bigrams (uni-grams) in are masked; the specific process is:

[0058] For a sentence sample w={w1,w1,…,w j ,…,w n}, if neither of the two unigrams in the bi-gram is masked, then the masking score M(w j ,D s ,D t ), if the sum of the masking scores of the two unigrams is greater than the threshold t, the two unigrams are masked; if the sum of the masking scores of the two unigrams is less than or equal to the threshold t, the two unigrams are not masked;

[0059] If one or both of the two unigrams in a bigram are masked, the bigram is not masked.

[0060] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0061] Specific implementation method 5: This implementation method is different from the specific implementation methods 1 to 4 in that the training set D with emotion labels in the source domain in 4) s All uni-grams in are masked; the specific process is:

[0062] For a sentence sample w={w1,w1,…,w j ,…,w n}, if none of the three single words in the trigram are masked, then the masking score M(w j ,D s ,D t ), if the sum of the masking scores of the three unigrams is greater than the threshold t, the three unigrams are masked; if the sum of the masking scores of the three unigrams is less than or equal to the threshold t, the three unigrams are not masked;

[0063] If one, two or three of the three unigrams in a trigram are masked, the trigram is not masked.

[0064] For example, the unigrams "paper" and "towel" have weak relevance to the Kitchen domain and are therefore not masked as unigrams. However, the bigram "papertowel" as a bigram is strongly relevant to the Kitchen domain and its masking score is above the threshold, so the bigram is masked. The rationale behind this higher-order n-gram masking approach is to capture the context more accurately. Higher-order terms such as bigrams and trigrams provide richer contextual information than unigrams. By masking bigrams and trigrams, we ensure that domain-specific phrases are identified while still allowing individual words to be used in other, potentially irrelevant contexts. This approach prevents the loss of useful words that might be unnecessarily masked if only higher-dimensional terms are considered.

[0065] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0066] Specific implementation method 6: This implementation method is different from any one of the specific implementation methods 1 to 5 in that the masking score M(w j ,D s ,D t ) is calculated as follows:

[0067] 1) Assume that the prior probability of each field is equal and use Bayes' rule;

[0068] Calculate the word w in the sentence sample j Belongs to the source domain sample set D s The probability P(D s w j ), the expression is:

[0069]

[0070] in, Denotes the source domain sample set D s The total number of samples included in ;

[0071] Denotes the source domain sample set D s The number of samples of domain-related terms included in the book (a book in the book's domain is a good domain-related term, and a book in the book's domain is not a good domain-related term);

[0072] α represents a smoothing hyperparameter, and ∝ represents correlation;

[0073] 2) Calculate the word w in the sentence sample j Belonging to the target domain sample set D t The probability P(D t w j ), the expression is:

[0074]

[0075] in, Denotes the target domain sample set D t The total number of samples included in ;

[0076] Denotes the target domain sample set D t The number of samples of domain-related terms included in the book (a book in the book's domain is a good domain-related term, and a book in the book's domain is not a good domain-related term);

[0077] α represents a smoothing hyperparameter, and ∝ represents correlation;

[0078] 3) Based on P(D s w j ) Calculate the entropy H(D s w j );

[0079] 4) Based on P(D t w j ) Calculate the entropy H(D t w j );

[0080] To effectively identify domain-specific terms w, we need a metric that captures both the likelihood of a term belonging to a domain and its specificity to that domain.

[0081] 5) Based on P(D s w j ) and H(D s w j ), calculate the word w in the sentence sample j and source domain sample set D s The correlation between ρ(w j ,D s ); the expression is:

[0082]

[0083] Where N represents the number of unlabeled fields;

[0084] logN is the entropy H(D s w j ) is the upper bound of ; higher entropy values ​​indicate that word w j It is not related to any particular field;

[0085] 6) Based on P(D t w j ) and H(D t w j), calculate the word w in the sentence sample j and the target domain sample set D t The correlation between ρ(w j ,D t ); the expression is:

[0086]

[0087] Where N represents the number of unlabeled fields;

[0088] logN is the entropy H(D t w j ) is the upper bound of ; higher entropy values ​​indicate that word w j It is not related to any particular field;

[0089] 7) Based on the word w in the sample j and source domain sample set D s The correlation between ρ(w j ,D s ), and the word w in the sample j and the target domain sample set D t The correlation between ρ(w j ,D t ), calculate the word w in the sample j In the source domain, the source domain sample set D s and the target domain sample set D t The masking score M(w j ,D s ,D t ):

[0090] M(w j ,D s ,D t )=ρ(w j ,D s )-ρ(w j ,D t )

[0091] ρ(w j ,D s ) represents the word w in the sample j and source domain sample set D s The relationship between

[0092] ρ(w j ,D t ) represents the word w in the sample j and the target domain sample set D t The relationship between

[0093] M() can take negative values ​​to prevent masking of n-grams that should be included in the original target domain samples.

[0094] M(w j ,D s ,D t )=ρ(w j ,D s )-ρ(w j ,D t ) is used to more accurately identify domain-related words, especially those words that appear frequently in both the source and target domains but do not have obvious domain relevance.

[0095] By calculating ρ(w j ,D s ) and ρ(w j ,D t ), that is, to measure the difference between word w j In the source domain sample set D s and the target domain sample set D t The association degree in , thereby determining the word w j Whether it has unique importance in a specific field. This difference helps identify words that appear frequently in both fields but are not field-specific, and exclude them from field-related words.

[0096] M(w j ,D s ,D t )=ρ(w j ,D s )-ρ(w j ,D t ) is designed to enhance the recognition accuracy of domain-related words. j By comparing the correlation between the source domain and the target domain, this formula can effectively prevent high-frequency words that frequently appear in both domains from being misjudged as domain-specific words. Especially in natural language processing, many words are frequently used in various fields but do not have domain-specificity. The mask score M(w j ,D s ,D t ), the model can more accurately identify words with unique meanings in a specific field, thereby improving the recognition effect of field-related words.

[0097] The other steps and parameters are the same as those in Specific Implementations 1 to 5.

[0098] Specific implementation method 7: This implementation method is different from one of the specific implementation methods 1 to 6 in that the domain reconstruction in step 1 and 2 is: based on the test set samples w in the target domain without labeled emotion labels t and the training set D with emotion labels in the source domain after maskings , get the original target domain guide sample w r ; The specific process is:

[0099] Step 121

[0100] The training set D with emotion labels in the masked source domain obtained in step one by one s Denoted as M(w r );

[0101] The test set D that is not labeled with sentiment labels in the target domain t The field direction vector v t and M(w r )’s embedding vector v r Concatenate along the feature dimension to generate a cascade matrix;

[0102] The specific process is:

[0103] 1) Design a learnable embedding matrix, which contains K×V sized embedding vectors as M(w r )’s embedding vector v r , allowing each field to be represented by K different vectors; K represents the number of vectors, and V represents the dimension of the vector;

[0104] 1≤K≤100 (K represents the number of words), V is 128;

[0105] 2) Use the domain name (such as kitchen) and domain words (such as cutting board, kitchen knife, etc.) to initialize the domain direction vector as the test set D of the target domain without emotional labels. t The field direction vector v t ;

[0106] 3) The test set D without emotion labels in the target domain t The field direction vector v t and M(w r )’s embedding vector v r Concatenate along the feature dimension to generate a cascade matrix;

[0107] Step 122: Input the cascade matrix into the T5 generative model based on the encoder-decoder architecture. The T5 generative model generates the original target domain guide sample w r ;

[0108] Each original target domain guide sample w r Corresponding to a single sample w s , w r and w s Share the same sentiment label (different domains, same sentiment label).

[0109] These samples are used to conditionally guide the diffusion model.

[0110] It is worth noting that, according to the requirements of this work, these initialized direction vectors gradually converge to different valid values ​​during the training process.

[0111] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0112] Specific implementation eight: This implementation differs from any one of specific implementations one to seven in that the counterfactual target domain sample generation layer is used to convert the training set D with emotion labels in the source domain into s And the generated original target domain guided sample set D r The diffusion model is input in the form of sample pairs, and the diffusion model generates a qualified counterfactual target domain sample set D c ; The specific process is:

[0113] Diffusion model prerequisites:

[0114] The diffusion model is a latent variable model originally designed for continuous regions;

[0115] The diffusion model includes two processes: forward diffusion process and reverse diffusion process;

[0116] In the forward process, given a sample x0, taken from the distribution q(x0), the Markov chain of latent variables x1...x T is generated by gradually adding Gaussian noise:

[0117]

[0118] Among them, β t is the noise schedule that controls the step size of noise addition. Finally, x T Approximately an isotropic Gaussian distribution. If β t is small enough, then the reverse process qx t-1 ∣x t also follows a Gaussian distribution and can be modeled by the following formula:

[0119]

[0120] Among them, μ θ (·) and Σ θ (·) This can be implemented using U-Net or Transformer.

[0121] Diffusion models have emerged as a new paradigm for generative models compared to traditional generative models such as generative adversarial networks (GANs). They have several potential advantages, especially in generating high-quality text and images. However, most current diffusion works face challenges in terms of training convergence and generation speed, especially given that these models need to use minimum Bayesian risk (MBR) strategies for decoding and generation, resulting in significant computational overhead during training. Moreover, in domain adaptation, there are concerns about the quality of generated target domain samples in low-resource settings, especially due to the failure of data distribution shift.

[0122] Step 21: Diffusion model training phase; the specific process is:

[0123] In order to ensure the quality of generated samples, we introduced the soft absorbing state (SAS) and the maximum mean discrepancy loss (MMD) in the training stage, which helps the diffusion model learn the ability to reconstruct discrete mutations based on the underlying Gaussian space, thereby enhancing its ability to recover conditional signals; at the same time, in the real target domain data D t Under the supervision of c From the source domain D s Transition to target domain D t ;

[0124] 1) In the diffusion model, the samples w in the training set with emotion labels in the source domain are s and the original target domain guided sample set sample w r Connect to get samples

[0125] 2) Let time step t = 0;

[0126] 3) Use x in the diffusion model t express The potential representation of; the specific process is:

[0127] The sample Embedded into a continuous feature space, the forward process of the diffusion model is obtained;

[0128]

[0129] Among them, I is a unit matrix; N() represents the normal distribution; β0 represents the scheduling coefficient of the noise variance, Indicates that the sample Embed; x t represents the perturbation data after the forward noise addition process; Indicates that the samples w in the training set with emotion labels in the source domain are s and the original target domain guided sample set sample w r The sample after connection; represents the forward process of the diffusion model;

[0130] In the initial step of the forward noise addition process, the discrete samples w s Mapping to continuous vector space;

[0131] 4) Based on x t Infer the latent representation x at time step t t as follows:

[0132]

[0133] in, represents the noise attenuation coefficient at the tth time step in the diffusion model; ∈ represents the noise of normal distribution;

[0134]

[0135] in, and Represent the source domain samples w s and the original target domain sample w r potential representation of;

[0136] 5) The potential representation x at time step t t The i-th token in is replaced by the soft absorbing state n;

[0137] The soft absorbing state n has the same dimension as the token;

[0138]

[0139] in, represents the i-th token at the t-th time step after replacement; Denotes the potential representation x at time step t t The i-th token in ; η represents the mark for judging whether to perform soft absorbing state replacement;

[0140] η=Bernoulliβ t *γ, β t represents the scheduling coefficient of the noise variance at the t-th step, Bernoulli represents the Bernoulli equation, and * represents the multiplication sign; γ is the ratio of the words in the sentence being masked [MASK] at t = T;

[0141] The introduction of soft absorbing states enhances the model's ability to handle discrete data in the continuous diffusion process;

[0142] At the same time, soft constraints are provided in the high-dimensional feature space, which enhances the stability and reliability of the model. In addition, t Different from the traditional diffusion model, Replace with Come only to Introducing some noise. This is the key aspect that enables diffusion models to perform conditional language modeling.

[0143] In the reverse process, the goal is to remove the Gaussian noise from the part by jointly denoising and discretizing the noise. Restore to initial state;

[0144] 6) The overall objective function is derived by adding the two components:

[0145]

[0146] in, It is a weight parameter that starts from zero and gradually increases during model training to ensure a balance between reconstruction ability and distribution transfer ability throughout the training process;

[0147] is the loss function of the diffusion model; is the MMD loss function;

[0148] The MMD loss function Expressed as

[0149]

[0150] Among them, N C represents the total number of samples in the sample set of the counterfactual target domain;

[0151] N t represents the total number of samples in the test set that are not labeled with sentiment labels in the target domain;

[0152] k() represents the Gaussian kernel function;

[0153] Expressing sentences and sentences The square distance between

[0154]

[0155] To ensure that the data distribution is derived from the counterfactual target sample w in Dc c Transition to D t The real target domain sample w in t , the following sentence-level MMD loss is proposed:

[0156] When the MMD loss is minimum, Dc The distribution is close to D t distribution, thereby improving the quality of generated samples;

[0157] Step 2: Sampling stage (SampleStage) of the diffusion model. The specific process is as follows:

[0158] Previously, the diffusion model used rounding operations in the sampling phase to predict vectors and reduce rounding errors. However, the difference between training and sampling may lead to the accumulation of prediction errors and a decrease in sampling speed. In order to improve the sampling speed of the diffusion model, we used the advanced ordinary differential equation accelerator DPM-Solver++ as a sampling accelerator for continuous space in the sampling phase. This accelerator does not require minimum Bayesian risk (MBR) decoding during the sampling process, thus saving a lot of time. Importantly, it improves the sampling speed while also ensuring the quality of the generated samples.

[0159] Specifically, The definition adds discrete noise to the continuous Gaussian noise to build a bridge between training and reasoning in discrete space.

[0160] Using the exact solution of the diffusion equation proposed by DPM-Solver++, set the initial value of the time step s, s>0;

[0161] The potential representation at time step s is x s ;

[0162] Based on the initial value x s Get the potential representation x at time step t t , time t∈[0,s]:

[0163]

[0164] Among them, λ t is a strictly decreasing function of t;

[0165] σ t is the coefficient related to the noise at time step t, σ s is the coefficient related to the time step s and the noise;

[0166] f θ is the output of the diffusion model at time step t;

[0167] represents the potential vector in the soft absorption state, λ represents the time variable used to simplify the noise control process;

[0168] λ t represents the variable about time step t used to simplify the noise control process, λs represents the variable with respect to time step s used to simplify the noise control process;

[0169] σ t Relative to β t is monotonic, and f θ Used to corrupt data x t Restore the data prediction model to x0;

[0170] In addition, the formula requires Approximation. The integral can be calculated analytically by repeatedly applying block n integrals, and only the first few terms can be approximated, while higher-order error terms are ignored. In our experiments, we use the second order. After discrete denoising in our method, the algorithm is still applicable because Consistent with training objectives.

[0171] Step 23: Set time step t=t+1; repeat step 21 and step 22 until the overall objective function converges to obtain a trained diffusion model;

[0172] Step 24: The training set D with emotion labels in the source domain s And the generated original target domain guided sample set D r The trained diffusion model is input in the form of sample pairs, and the trained diffusion model generates a qualified counterfactual target domain sample set D c .

[0173] On this basis, using the source domain dataset D s and the corresponding generated counterfactual target domain dataset D c Train the classifier, where D c The sample labels in D s and D r to predict the labels of test samples from the target domain.

[0174] Stop training and sampling after 2000 time steps to get D c , source domain dataset D s and the corresponding generated counterfactual target domain dataset D c Input the classifier and train the classifier until convergence to obtain a trained classifier.

[0175] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0176] Specific embodiment 9: This embodiment is different from any one of specific embodiments 1 to 8 in that the target domain sample data classification layer is used to train the Bert classification model using samples labeled with emotion and with domain, respectively, and the trained Bert classification model is used to classify the test set D of the target domain without labeled emotion labels. t Perform sentiment classification; including the following steps:

[0177] Step 31: Get the trained emotional Bert classification model; the specific process is:

[0178] The trained diffusion model generates a qualified counterfactual target domain sample set D c and the training set D with sentiment labels in the source domain s Input the Bert classification model, train the Bert classification model, and obtain the trained emotional Bert classification model;

[0179] Step 3.2: Test set D of unlabeled sentiment labels in the target domain t Input the trained emotional Bert classification model, and the trained emotional Bert classification model outputs the test set samples w in the target domain with unlabeled emotional labels. t Sentiment classification results.

[0180] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0181] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement, characterized by: The specific process of the method is: Step 1: Obtain sample data in the sentiment dataset; The sample data in the sentiment dataset are all sentiment-labeled data; The sample data in the sentiment dataset carry different types of sentiment labels; The sample data in the sentiment dataset is divided into a training set and a test set; The training set and test set come from different fields; The test set sentiment labels are invisible to the model; The training set and test set include samples w and sentiment s; Step 2: Create CDA 2 Model; CDA 2 The model includes: original target domain sample generation layer, counterfactual target domain sample generation layer, and target domain sample data classification layer; The original target domain sample generation layer transforms the sample w in the training set with emotion labels in the source domain into s and the unlabeled sentiment labels of the target domain in the test set w t Generate original target domain bootstrap samples w r ; The counterfactual target domain sample generation layer is used to convert the training set D with emotion labels in the source domain s And the generated original target domain guided sample set D r The diffusion model is input in the form of sample pairs, and the diffusion model generates a qualified counterfactual target domain sample set D C ; A test set D of unlabeled sentiment labels of the target domain t The unlabeled sentiment labels of the target domain in the test set are samples w t composition; The counterfactual target domain sample set D C From the counterfactual target domain sample set, sample w c composition; The original target domain guide sample set D r The sample w in the sample set is guided by the original target domain r composition; The target domain sample data classification layer is used to train the Bert classification model using samples labeled with emotion and domain respectively, and the trained Bert classification model is used to classify the test set D of the target domain without labeled emotion labels. t Perform sentiment classification.

2. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 1 is characterized by: The original target domain sample generation layer transforms the samples w in the training set with emotion labels in the source domain into s and the unlabeled sentiment labels of the target domain in the test set w t Generate original target domain bootstrap samples w r ; The specific process is: Step 1. Domain destruction: training set D with sentiment labels in the source domain s The specific domain terms in the source domain are masked to obtain the training set D with sentiment labels after masking. s ; The specific process is: 1) Training set D s It includes sentence samples, and sentences include words; Let w = {w1,w1,…,wj,…,wn} represent the training set D with emotion labels in the source domain s A sample of sentences of length n in , w j Represents a word in a sentence sample; 2) Training set D with emotion labels in the source domain s Mask all unigrams in ; 3) Training set D with emotion labels in the source domain s Mask all bigrams in ; 4) Training set D with emotion labels in the source domain s Mask all the tri-grams in the source domain to obtain the masked training set D with sentiment labels. s ; Step 1 and 2: Domain reconstruction: Samples w in the test set with unlabeled sentiment labels based on the target domain t and the training set D with emotion labels in the source domain after masking s , get the original target domain guide sample w r .

3. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 2 is characterized in that: The training set D with emotion labels for the source domain in 2) s Mask all unigrams in ; the specific process is: Calculate the masking score M(w j ,D s ,D t ), if the masking score of a unigram is greater than the threshold t, the unigram is masked; if the masking score of a unigram is less than or equal to the threshold t, the unigram is not masked.

4. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 3 is characterized by: The training set D with emotion labels for the source domain in 3) s Mask all the bigrams in ; the specific process is: For a sentence sample w={w1,w1,…,w j ,…,w n }, if neither of the two unigrams in the bigram is masked, then the masking score M(w j ,D s ,D t ), if the sum of the masking scores of the two unigrams is greater than the threshold t, the two unigrams are masked; if the sum of the masking scores of the two unigrams is less than or equal to the threshold t, the two unigrams are not masked; If one or both of the two unigrams in a bigram are masked, the bigram is not masked.

5. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 4 is characterized in that: The training set D with emotion labels for the source domain in 4) s Mask all tri-grams in ; the specific process is: For a sentence sample w={w1,w1,…,w j ,…,w n }, if none of the three single words in the trigram are masked, then the masking score M(w j ,D s ,D t ), if the sum of the masking scores of the three unigrams is greater than the threshold t, the three unigrams are masked; if the sum of the masking scores of the three unigrams is less than or equal to the threshold t, the three unigrams are not masked; If one, two or three of the three unigrams in a trigram are masked, the trigram is not masked.

6. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 5 is characterized by: The masking score M(w j ,D s ,D t ) is calculated as follows: 1) Calculate the word w in the sentence sample j Belongs to the source domain sample set D s The probability P ( D s |w j) , the expression is: in, Denotes the source domain sample set D s The total number of samples included in ; Denotes the source domain sample set D s The number of samples of domain terms included in ; α represents a smoothing hyperparameter, and ∝ represents correlation; 2) Calculate the word w in the sentence sample j Belonging to the target domain sample set D t The probability P ( D t |w j) , the expression is: in, Denotes the target domain sample set D t The total number of samples included in ; Denotes the target domain sample set D t A sample of the domain terms included in; α represents a smoothing hyperparameter, and ∝ represents correlation; 3) Based on P ( D s |w j) Calculate entropy H ( D s |w j) ; 4) Based on P ( D t |w j) Calculate entropy H ( D t |w j) ; 5) Based on P ( D s |w j) and H ( D s |w j) , calculate the word w in the sentence sample j and source domain sample set D s The correlation between ρ(w j ,D s ); the expression is: Where N represents the number of unlabeled fields; 6) Based on P ( D t |w j) and H ( D t |w j) , calculate the word w in the sentence sample j and the target domain sample set D t The correlation between ρ(w j ,D t ); the expression is: Where N represents the number of unlabeled fields; 7) Based on the word w in the sample j and source domain sample set D s The correlation between ρ(w j ,D s ), and the word w in the sample j and the target domain sample set D t The correlation between ρ(w j ,D t ), calculate the word w in the sample j In the source domain, the source domain sample set D s and the target domain sample set D t The masking score M(w j ,D s ,D t ): M(w j ,D s ,D t )=ρ(w j ,D s )-ρ(w j ,D t ) ρ(w j ,D s ) represents the word w in the sample j and source domain sample set D s The relationship between ρ(w j ,D t ) represents the word w in the sample j and the target domain sample set D t The connection between them.

7. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 6 is characterized in that: Domain reconstruction in steps 1 and 2: Samples w in the test set with unlabeled sentiment labels based on the target domain t and the training set D with emotion labels in the source domain after masking s , get the original target domain guide sample w r ; The specific process is: Step 121 The training set D with emotion labels in the masked source domain obtained in step one by one s Denoted as M(w r ); The test set D that is not labeled with sentiment labels in the target domain t The field direction vector v t and M(w r )’s embedding vector v r Concatenate along the feature dimension to generate a cascade matrix; The specific process is: 1) Design a learnable embedding matrix, which contains K×V sized embedding vectors as M(w r )’s embedding vector v r ; K represents the number of vectors, V represents the dimension of the vector; 2) Use the domain name and domain words to initialize the domain direction vector as the test set D of the target domain without labeled sentiment labels t The field direction vector v t ; 3) The test set D without emotion labels in the target domain t The field direction vector v t and M(w r )’s embedding vector v r Concatenate along the feature dimension to generate a cascade matrix; Step 122: Input the cascade matrix into the T5 generative model based on the encoder-decoder architecture. The T5 generative model generates the original target domain guide sample w r ; Each original target domain guide sample w r Corresponding to a single sample w s , w r and w s share the same sentiment label.

8. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 7 is characterized in that: The counterfactual target domain sample generation layer is used to transform the training set D with emotion labels in the source domain s And the generated original target domain guided sample set D r The diffusion model is input in the form of sample pairs, and the diffusion model generates a qualified counterfactual target domain sample set D c ; The specific process is: Step 21: Diffusion model training phase; the specific process is: 1) In the diffusion model, the samples w in the training set with emotion labels in the source domain are s and the original target domain guided sample set sample w r Connect to get samples 2) Let time step t = 0; 3) Use x in the diffusion model t express potential representation of; The specific process is: The sample Embedded into a continuous feature space, the forward process of the diffusion model is obtained; Among them, I is a unit matrix; N() represents the normal distribution; β0 represents the scheduling coefficient of the noise variance, Indicates that the sample Embed; x t represents the perturbation data after the forward noise addition process; Indicates that the samples w in the training set with emotion labels in the source domain are s and the original target domain guided sample set sample w r The sample after connection; represents the forward process of the diffusion model; 4) Based on x t Infer the latent representation x at time step t t as follows: in, represents the noise attenuation coefficient at the tth time step in the diffusion model; ∈ represents the noise of normal distribution; 5) The potential representation x at time step t t The i-th token in is replaced by the soft absorbing state n; The soft absorbing state n has the same dimension as the token; in, represents the i-th token at the t-th time step after replacement; represents the potential representation x at time step t t The i-th token in ; η represents the mark for judging whether to perform soft absorbing state replacement; 6) The overall objective function is derived by adding the two components: in, is the weight parameter; is the loss function of the diffusion model; is the MMD loss function; The MMD loss function Expressed as Among them, N C represents the total number of samples in the sample set of the counterfactual target domain; N t represents the total number of samples in the test set that are not labeled with sentiment labels in the target domain; k() represents the Gaussian kernel function; Expressing sentences and sentences The square distance between Step 2: Sampling stage of diffusion model. The specific process is as follows: Set the initial value of the time step s, s>0; The potential representation at time step s is x s ; Based on the initial value x s Get the potential representation x at time step t t , time t∈[0,s]: Among them, σ t is the coefficient related to the noise at time step t, σ s is the coefficient related to the time step s and the noise; f θ is the output of the diffusion model at time step t; represents the latent vector in the soft absorption state, and λ represents the variable about time; λ t represents the variable at time step t, λ s represents the variable with respect to time step s; Step 23: Set time step t=t+1; repeat step 21 and step 22 until the overall objective function converges to obtain a trained diffusion model; Step 24: The training set D with emotion labels in the source domain s And the generated original target domain guided sample set D r The trained diffusion model is input in the form of sample pairs, and the trained diffusion model generates a qualified counterfactual target domain sample set D c .

9. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 8 is characterized by: The target domain sample data classification layer is used to train the Bert classification model using samples labeled with emotion and with domain, respectively, and use the trained Bert classification model to classify the test set D of the target domain without labeled emotion labels. t Perform sentiment classification; including the following steps: Step 31: Get the trained emotional Bert classification model; the specific process is: The trained diffusion model generates a qualified counterfactual target domain sample set D c and the training set D with sentiment labels in the source domain s Input the Bert classification model, train the Bert classification model, and obtain the trained emotional Bert classification model; Step 3.2: Test set D of unlabeled sentiment labels in the target domain t Input the trained emotional Bert classification model, and the trained emotional Bert classification model outputs the test set samples w in the target domain with unlabeled emotional labels. t Sentiment classification results.

Images