Sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement

Abstract

This invention relates to a sentiment analysis method based on low-resource cross-domain diffusion counterfactual data augmentation. It is applicable to the fields of artificial intelligence, natural language processing, and text generation technology, specifically text sentiment classification methods. This invention addresses the problem that current sentiment analysis methods cannot effectively handle cross-domain data due to the model being trained on in-domain data. The process involves establishing a CDA (Content Assisted Analysis) system. 2 The model consists of: a primary target domain sample generation layer that generates primary target domain guidance samples from samples in the training set with sentiment labels from the source domain and samples in the test set with unlabeled sentiment labels from the target domain; a counterfactual target domain sample generation layer that inputs the training set with sentiment labels from the source domain and the generated primary target domain guidance sample set into the diffusion model as sample pairs, and the diffusion model generates a qualified counterfactual target domain sample set; and a target domain sample data classification layer that performs sentiment classification on the test set with unlabeled sentiment labels from the target domain.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, natural language processing, and text generation technology, and in particular to a text sentiment classification method. Background Technology

[0002] Sentiment analysis is a crucial task in Natural Language Processing (NLP), its core being the extraction of implicit sentiment from textual data. In recent years, its popularity has surged due to its widespread real-world applications. Deep learning techniques have made significant strides and achieved remarkable success in sentiment analysis. However, the performance of traditional sentiment analysis methods, which rely on labeled data to train models in the target domain, degrades significantly when operating under low-resource conditions or encountering data distribution shifts between the training and target domains. To reduce reliance on labeled data, cross-domain sentiment analysis has attracted considerable attention. Many previous works have resorted to unsupervised domain adaptation techniques, aiming to transfer knowledge from a resource-rich source domain to a target domain with unlabeled data. In cross-domain sentiment analysis tasks, most existing domain adaptation methods employ adversarial training to prevent the model from distinguishing domain-specific samples, thereby transferring knowledge from the source domain to the target domain and attempting to learn domain-specific knowledge. While these methods have achieved promising results, their models are only trained on in-domain labeled data in the source domain, limiting their ability to handle out-of-domain (OOD) data. To address these limitations, researchers have attempted to design cross-domain data augmentation methods. The key objective is to generate a large number of labeled target domain samples based on labeled source domain samples, thereby achieving knowledge transfer. Research within this framework primarily includes two methods: Masked Language Models (MLM) and Sequence-to-Sequence (Seq2Seq) models. The core idea of ​​both methods is to mask domain-specific terms in source domain samples and then use a trained MLM or Seq2Seq model to fill in the masked terms, thus achieving word replacement in the target domain. While word replacement-based data augmentation methods have shown advantages over feature adaptation methods, they still have some drawbacks: (i) the expanded samples generated through word replacement may lead to semantic breaks; (ii) they preserve the fixed syntactic structure of the source domain, establishing false associations between the generated text and the source domain; and (iii) the diversity of the generated samples is limited by the source domain template. Summary of the Invention

[0003] The purpose of this invention is to address the problem that current sentiment analysis methods cannot effectively handle out-of-domain (OOD) data due to the fact that the models are trained on in-domain (IID) data. Therefore, this invention proposes a sentiment analysis method based on low-resource cross-domain diffusion counterfactual data augmentation.

[0004] The specific process of the sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement is as follows:

[0005] Step 1: Obtain sample data from the sentiment dataset;

[0006] The sample data in the sentiment dataset are all labeled with sentiment.

[0007] The sample data within the sentiment dataset carries different types of sentiment labels;

[0008] The sample data in the sentiment dataset is divided into a training set and a test set;

[0009] The training and test sets come from different domains;

[0010] The sentiment tags on the test set are not visible to the model;

[0011] The training and test sets include samples w and sentiment s;

[0012] Step 2: Establish CDA 2 Model;

[0013] CDA 2 The model includes: a raw target domain sample generation layer, a counterfactual target domain sample generation layer, and a target domain sample data classification layer;

[0014] The original target domain sample generation layer, under a masking mechanism, generates samples from the sentiment-labeled training set in the source domain through domain destruction and reconstruction. s And the unlabeled sentiment tags in the test set of the target domain w t Generate original target domain guide samples w r ;

[0015] The counterfactual target domain sample generation layer is used to generate samples from the source domain's sentiment-labeled training set D. s and the generated original target domain guiding sample set D r The sample pairs are input into the diffusion model, which generates a qualified counterfactual target domain sample set D. C ;

[0016] The test set D of the target domain without labeled sentiment tags t Samples w from the test set of unlabeled sentiment data in the target domain t composition;

[0017] The counterfactual target domain sample set D C Sample w from the counterfactual target domain sample set c composition;

[0018] The original target domain guiding sample set D r Sample w is guided by the original target domain in the sample set. r composition;

[0019] The target domain sample data classification layer is used to train the BERT classification model using samples labeled with sentiment and samples labeled with domain, respectively. The trained BERT classification model is then used to test the unlabeled sentiment data set D in the target domain. t Perform sentiment categorization.

[0020] The beneficial effects of this invention are as follows:

[0021] This invention proposes a cross-domain adaptive framework, CDA, for low-resource sentiment analysis. 2 This framework utilizes counterfactual diffusion enhancement. CDA 2 This invention aims to mitigate semantic disruptions and spurious associations caused by fixed syntactic structures in the source domain. The method presented in this invention excels in generating counterfactual target domain samples, exhibiting outstanding diversity and fluency, unlike previous research. The framework constructs a simple word substitution method by implementing conditional guidance for the diffusion generator. Specifically, high-quality original target samples are provided to the generator through domain destruction and domain reconstruction. Next, during the training phase of the diffusion model, a learnable soft absorption state is designed by introducing additional discrete noise during the continuous diffusion process to better adapt to the inherent discrete nature of the text. Furthermore, a maximum mean deviation (MMD) loss is introduced, utilizing unlabeled samples from the real target domain to supervise the generation process, thereby facilitating better data distribution transfer. In the sampling phase, an advanced ordinary differential equation (ODE) accelerator is used to accelerate sampling while minimizing the loss of sample quality, thus generating high-quality counterfactual target samples. Attached Figure Description

[0022] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0023] Specific Implementation Method 1: This implementation method is based on a sentiment analysis method enhanced by low-resource cross-domain diffusion counterfactual data. The specific process is as follows:

[0024] Step 1: Obtain sample data from the sentiment dataset;

[0025] The sample data in the sentiment dataset are all labeled with sentiment.

[0026] The sample data in the sentiment dataset carries different types of sentiment labels (types of sentiment, positive or negative);

[0027] The sample data in the sentiment dataset is divided into a training set and a test set;

[0028] The training and test sets come from different domains (the types of domains, kitchen or books);

[0029] The sentiment tags on the test set are not visible to the model;

[0030] The training and test sets include samples w and sentiment s;

[0031] For example: samples w from the source domain with sentiment labels s "This book's journey fills readers with joy and anticipation," and includes text from the target domain related to kitchen appliances without emotional labeling. t As a CDA 2 The model's input is generated text w that exhibits the distribution characteristics of text data in the target domain. c It is CDA 2 The model's output;

[0032] Step 2: Establish CDA 2 Model;

[0033] CDA 2 The model includes: a raw target domain sample generation layer, a counterfactual target domain sample generation layer, and a target domain sample data classification layer;

[0034] The original target domain sample generation layer, under a masking mechanism, generates samples from the sentiment-labeled training set in the source domain through domain destruction and reconstruction. s And the unlabeled sentiment tags in the test set of the target domain w t Generate original target domain guide samples w r ;

[0035] The counterfactual target domain sample generation layer is used to generate samples from the source domain's sentiment-labeled training set D. s and the generated original target domain guiding sample set D r The sample pairs are input into the diffusion model, which generates a qualified counterfactual target domain sample set D. C ;

[0036] The test set D of the target domain without labeled sentiment tags t Samples w from the test set of unlabeled sentiment data in the target domain t composition;

[0037] The counterfactual target domain sample set D CSample w from the counterfactual target domain sample set c composition;

[0038] The original target domain guiding sample set D r Sample w is guided by the original target domain in the sample set. r composition;

[0039] The target domain sample data classification layer is used to train the BERT classification model using samples labeled with sentiment and samples labeled with domain, respectively. The trained BERT classification model is then used to test the unlabeled sentiment data set D in the target domain. t Perform sentiment categorization.

[0040] Further improve the sentiment and domain performance of the generated counterfactual target domain samples;

[0041] Problem definition:

[0042] This invention primarily studies cross-domain sentiment classification in low-resource scenarios. Two domains are considered: a source domain and a target domain. The source domain set D... s It contains both labeled and unlabeled data, with the amount of labeled data being significantly larger than the amount of unlabeled data. Target domain set D t Including a set of unlabeled data, the goal of cross-domain sentiment classification is to utilize D s and D t Predict the labels of test samples in the target domain.

[0043] The CDA 2 for CounterfactualDiffusionAugmentationforCross-DomainAdaptationin Low-ResourceSentimentAnalysis.

[0044] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the original target domain sample generation layer, under a masking mechanism, uses domain destruction and domain reconstruction to generate samples w from the sentiment-labeled training set of the source domain. s And the unlabeled sentiment tags in the test set of the target domain w t Generate original target domain guide samples w r ;

[0045] The specific process is as follows:

[0046] Step 11, Domain Destruction: This involves modifying the sentiment-labeled training set D from the source domain. s By masking specific domain-related terms, a sentiment-labeled training set D from the source domain is obtained. s The specific process is as follows:

[0047] 1) Training set D s The document includes sentence samples, and each sentence contains words;

[0048] Let w = {w1, w1, ..., wj, ..., wn} represent the sentiment-labeled training set D in the source domain. s A sample of sentences of length n, w j This represents a word in a sentence sample;

[0049] 2) The training set D with sentiment labels in the source domain s Mask all unigrams in the string;

[0050] 3) The training set D with sentiment labels in the source domain s Mask all unigrams in the string;

[0051] 4) The training set D with sentiment labels in the source domain s Mask all unigrams in the source domain to obtain the masked training set D with sentiment labels. s ;

[0052] Steps 1 and 2: Domain Reconstruction: Based on unlabeled sentiment tags in the test set of samples w from the target domain. t and the sentiment-labeled training set D of the masked source domain s The original target domain guided sample w is obtained. r .

[0053] The other steps and parameters are the same as in Specific Implementation Method 1.

[0054] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that, in step 2), the training set D with sentiment labels in the source domain... s Mask all unigrams in the string; the specific process is as follows:

[0055] Calculate the masking score M(w) for each unary word. j D s D t If the masking score of a unary word (a word in w) is greater than the threshold t, then the unary word is masked; if the masking score of a unary word is less than or equal to the threshold t, then the unary word is not masked.

[0056] The other steps and parameters are the same as in Specific Implementation Method 1.

[0057] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that, in step 3), the training set D with sentiment labels in the source domain...s Mask all unigrams in the string; the specific process is as follows:

[0058] For sentence samples w = {w1, w1, ..., w j ,…,w n For any two adjacent bigrams in w, if neither of the two unary words in the bigram is masked, then calculate the masking score M(w) for each unary word. j D s D t If the sum of the masking scores of two unary words is greater than the threshold t, then the two unary words are masked; if the sum of the masking scores of two unary words is less than or equal to the threshold t, then the two unary words are not masked.

[0059] If one or both unary words in a binary word are masked, then the binary word is not masked.

[0060] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0061] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that, in step 4), the training set D with sentiment labels in the source domain... s Mask all unigrams in the string; the specific process is as follows:

[0062] For sentence samples w = {w1, w1, ..., w j ,…,w n For any three adjacent bigrams in w, if none of the three unary words in the bigram are masked, then calculate the masking score M(w) for each unary word. j D s D t If the sum of the masking scores of the three unary words is greater than the threshold t, then the three unary words are masked; if the sum of the masking scores of the three unary words is less than or equal to the threshold t, then the three unary words are not masked.

[0063] If one, two, or all three unary words in a ternary word are masked, then the ternary word is not masked.

[0064] For example, the unary words “paper” and “towel” have a weak relevance to the Kitchen domain and are therefore not masked as unary words. However, the bigram “papertowel” is strongly relevant to the Kitchen domain as a bigram, so its masking score is above the threshold and it is masked. The basic principle behind this higher-order n-gram masking method is to capture context more accurately. Higher-order terms such as bigrams and trigrams provide richer contextual information compared to unary words. By masking bigrams and trigrams, we ensure that domain-specific phrases are identified while still allowing individual words to be used in other potentially irrelevant contexts. This approach prevents the loss of useful words that might be unnecessarily masked if only higher-dimensional terms are considered.

[0065] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0066] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that the masking fraction M(w) is... j D s D t The calculation process for ) is as follows:

[0067] 1) Assume that the prior probabilities of each domain are equal, and use Bayes' rule;

[0068] Calculate the word w in the sentence sample j Belonging to the source domain sample set D s The probability P(D) s w j The expression is:

[0069]

[0070] in, D represents the source domain sample set. s The total number of samples included;

[0071] D represents the source domain sample set. s The sample size of domain-related terms included in the book (the book's domain records "this book is good" as a domain-related term, while "she is good" is not a domain-related term);

[0072] α represents the smoothing hyperparameter, and ∝ represents the correlation;

[0073] 2) Calculate the word w in the sentence sample j Belonging to the target domain sample set D t The probability P(D) t w j The expression is:

[0074]

[0075] in, D represents the target domain sample set. t The total number of samples included;

[0076] D represents the target domain sample set. t The sample size of domain-related terms included in the book (the book's domain records "this book is good" as a domain-related term, while "she is good" is not a domain-related term);

[0077] α represents the smoothing hyperparameter, and ∝ represents the correlation;

[0078] 3) Based on P(D) s w j ) Calculate the entropy H(D) s w j );

[0079] 4) Based on P(D) t w j ) Calculate the entropy H(D) t w j );

[0080] In order to effectively identify domain-specific terms w, we need a metric to capture the likelihood that a term belongs to a particular domain and its specificity to that domain.

[0081] 5) Based on P(D) s w j ) and H(D s w j ), calculate the word w in the sentence sample j and source domain sample set D s The correlation between ρ(w) j D s The expression is:

[0082]

[0083] Where N represents the number of unlabeled domains;

[0084] logN is the entropy H(D) s w j The upper bound of ); a higher entropy value represents the word w j It is not related to any specific field;

[0085] 6) Based on P(D) t w j ) and H(D t w j), calculate the word w in the sentence sample j and the target domain sample set D t The correlation between ρ(w) j D t The expression is:

[0086]

[0087] Where N represents the number of unlabeled domains;

[0088] logN is the entropy H(D) t w j The upper bound of ); a higher entropy value represents the word w j It is not related to any specific field;

[0089] 7) Based on the word w in the sample j and source domain sample set D s The correlation between ρ(w) j D s ), and the word w in the sample j and the target domain sample set D t The correlation between ρ(w) j D t ), calculate the word w in the sample j In the source domain sample set D s and the target domain sample set D t The masking score M(w) j D s D t ):

[0090] M(w j D s D t )=ρ(w j D s )-ρ(w j D t )

[0091] ρ(w j D s ) represents the word w in the sample j and source domain sample set D s The relationship between them;

[0092] ρ(w j D t ) represents the word w in the sample j and the target domain sample set D t The relationship between them;

[0093] M() can be negative to prevent masking of n-grams that should be included in the original target domain sample.

[0094] M(w j D s D t )=ρ(w j D s )-ρ(w j D t The purpose of using ) is to more accurately identify domain-related words, especially those words that appear frequently in both the source and target domains but do not have obvious domain relevance.

[0095] By calculating ρ(w) j D s ) and ρ(w j D t The difference between the values ​​of the words w, i.e., the values ​​that measure the difference between the words w, are the values ​​that measure j In the source domain sample set D s and the target domain sample set D t The degree of relevance in the word w is used to determine the word w. j Does it possess unique importance within a specific domain? This difference helps identify words that appear frequently in both domains but are not domain-specific, excluding them from domain-related terms.

[0096] M(w j D s D t )=ρ(w j D s )-ρ(w j D t The design aims to enhance the accuracy of domain-related word recognition. This is achieved by analyzing the word w... j By comparing the relevance between the source and target domains, this formula effectively avoids misclassifying high-frequency words that appear frequently in both domains as domain-specific words. This is especially important in natural language processing, where many words are frequently used across various domains but lack domain-specific discrimination. The mask score M(w) obtained using the third formula... j D s D t The model can more accurately identify words with unique meanings in specific domains, thereby improving the recognition effect of domain-related words.

[0097] The other steps and parameters are the same as those in one of the specific implementation methods one to five.

[0098] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that the domain reconstruction in steps one and two involves: based on unlabeled sentiment tags in the test set samples w of the target domain. t and the sentiment-labeled training set D of the masked source domains The original target domain guided sample w is obtained. r The specific process is as follows:

[0099] Step 1, Step 2, Step 1

[0100] The masked source domain training set D obtained step by step is used to... s denoted as M(w) r );

[0101] The test set D, which is not labeled with sentiment in the target domain. t The domain direction vector v t With M(w) r The embedding vector v r Concatenate matrices by splicing along the feature dimensions;

[0102] The specific process is as follows:

[0103] 1) Design a learnable embedding matrix containing K×V size embedding vectors, where the K×V size embedding vectors are used as M(w r The embedding vector v r Each domain can be represented by K distinct vectors; K represents the number of vectors, and V represents the dimension of the vectors;

[0104] 1≤K≤100 (K represents the number of words), V takes the value 128;

[0105] 2) Initialize the domain orientation vector using domain names (e.g., kitchen) and domain terms (e.g., cutting board, knife, etc. in the kitchen), which will serve as the test set D for the target domain without sentiment labels. t The domain direction vector v t ;

[0106] 3) The test set D, which is not labeled with sentiment tags in the target domain. t The domain direction vector v t With M(w) r The embedding vector v r Concatenate matrices by splicing along the feature dimensions;

[0107] Step 122: Input the concatenated matrix into the T5 generative model based on the encoder-decoder architecture. The T5 generative model generates the original target domain guide sample w. r ;

[0108] Each original target domain guides samples w r Corresponding to a single sample w s w r and w s They share the same sentiment tags (even though they belong to different domains).

[0109] These samples are used to conditionally guide diffusion models.

[0110] It is worth noting that, according to the requirements of this work, these initialized direction vectors gradually converge to different effective values ​​during the training process.

[0111] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0112] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One through Seven in that the counterfactual target domain sample generation layer is used to generate the source domain training set D with sentiment labels. s and the generated original target domain guiding sample set D r The sample pairs are input into the diffusion model, which generates a qualified counterfactual target domain sample set D. c The specific process is as follows:

[0113] Prerequisites for diffusion models:

[0114] The diffusion model is a latent variable model originally designed for continuous regions;

[0115] The diffusion model includes two processes: forward diffusion and reverse diffusion.

[0116] In the forward process, given a sample x0, taken from the distribution q(x0), the Markov chain of latent variables x1...x T It was generated by gradually adding Gaussian noise:

[0117]

[0118] Where, β t It's noise scheduling that controls the step size of noise addition. Ultimately, x T It approximates an isotropic Gaussian distribution. If β t If it is small enough, then the reverse process qx t-1 |x t It also follows a Gaussian distribution and can be modeled by the following formula:

[0119]

[0120] Where, μ θ (·) and Σ θ (·) This can be achieved using U-Net or Transformer.

[0121] Diffusion models have emerged as a new paradigm for generative models compared to traditional generative models such as Generative Adversarial Networks (GANS). They offer several potential advantages, particularly in generating high-quality text and images. However, most current diffusion work faces challenges in training convergence and generation speed, especially considering that these models require the use of a minimum Bayesian risk (MBR) policy for decoding and generation, leading to significant computational overhead during training. Furthermore, in domain adaptation, there are concerns about the quality of target domain samples generated in low-resource environments, particularly due to failures in data distribution transitions.

[0122] Step 2.1: Training phase of the diffusion model; the specific process is as follows:

[0123] To ensure the quality of the generated samples, we introduced SoftAbsorbingState (SAS) and MaximumMeanDiscrepancy (MMD) during the training phase. This helps the diffusion model learn to reconstruct discrete mutations based on the underlying Gaussian space, thereby enhancing its ability to recover conditional signals. Simultaneously, in the real target domain data D... t Under supervision, MMD loss can improve the generation of sample w c From source domain D s Transition to target domain D t ;

[0124] 1) Within the diffusion model, the samples w in the training set with sentiment labels from the source domain are... s and the original target domain guided sample set sample w r Connect them to get the sample

[0125] 2) Set the time step t = 0;

[0126] 3) Using x in the diffusion model t express The potential representation; the specific process is as follows:

[0127] Sample Embedded into a continuous feature space, the forward process of the diffusion model is obtained;

[0128]

[0129] Where I is an identity matrix; N() represents a normal distribution; β0 represents the scheduling coefficient of the noise variance. Indicates that the sample Embedded; x t This represents the perturbed data after the forward noise addition process; This indicates that the training set samples w with sentiment labels from the source domain will be used. s and the original target domain guided sample set sample w r The connected samples; This represents the forward process of the diffusion model;

[0130] In the initial step of the forward noise addition process, the discrete sample w s Mapped to a continuous vector space;

[0131] 4) Based on x t Derive the latent representation x at time step t t as follows:

[0132]

[0133] in, denoted by , representing the noise attenuation coefficient at time step t in the diffusion model; ∈ represents normally distributed noise;

[0134]

[0135] in, and Representing source domain samples w respectively s and the original target domain sample w r The potential representation of;

[0136] 5) The latent representation x of time step t t The i-th token in the array is replaced with a soft-absorbing state n;

[0137] The soft-absorbing state n has the same dimension as the token;

[0138]

[0139] in, This represents the i-th token at time step t after the replacement; The latent representation x at time step t t The i-th token in the array; η represents the flag used to determine whether to perform a soft absorption state substitution;

[0140] η=Bernoulliβ t *γ,β t The scheduling coefficient represents the noise variance at step t, Bernoulli represents the Bernoulli equation, and * represents the multiplication sign; γ is the ratio of words in the sentence that are masked at t = T.

[0141] The introduction of soft absorption states enhances the model's ability to handle discrete data during continuous diffusion.

[0142] Simultaneously, soft constraints are provided in the high-dimensional feature space, enhancing the stability and reliability of the model. Furthermore, with the entire perturbation x... t Unlike traditional diffusion models, by... Replace with Come only to Introducing some noise is a key aspect that enables the diffusion model to perform conditional language modeling.

[0143] In the reverse process, the goal is to remove partial Gaussian noise by combining denoising and discrete noise. Restore to the initial state;

[0144] 6) The overall objective function is derived by adding the two components together:

[0145]

[0146] in, These are weight parameters that start from zero and gradually increase during model training to ensure a balance between reconstruction capability and distribution transfer capability throughout the training process;

[0147] This is the loss function for the diffusion model; MMD loss function;

[0148] The MMD loss function Represented as

[0149]

[0150] Where, N C This represents the total number of samples in the counterfactual target domain sample set;

[0151] N t This represents the total number of samples in the test set that are not labeled with sentiment tags in the target domain;

[0152] k() represents the Gaussian kernel function;

[0153] Sentence and sentences The squared distance between them;

[0154]

[0155] To ensure that the data distribution is derived from the counterfactual target sample w in DC c Transition to D t The real target domain sample w t The following statement-level MMD loss is proposed:

[0156] When MMD loss is minimized, Dc The distribution is close to D t The distribution of samples improves the quality of the generated samples.

[0157] Step 22, the sampling stage of the diffusion model, the specific process is as follows:

[0158] Previously, diffusion models used rounding operations during the sampling phase to predict vectors and reduce rounding errors. However, the difference between training and sampling can lead to the accumulation of prediction errors and a decrease in sampling speed. To improve the sampling speed of diffusion models, we used the advanced ordinary differential equation accelerator DPM-Solver++ as a continuous-space sampling accelerator during the sampling phase. This accelerator does not require minimum Bayes risk (MBR) decoding during sampling, thus saving significant time. Importantly, it improves sampling speed while maintaining the quality of generated samples.

[0159] Specifically, such as The definition adds discrete noise to continuous Gaussian noise, bridging training and inference in discrete space.

[0160] Using the exact solution of the diffusion equation proposed by DPM-Solver++, the initial value of the time step is set to s, where s>0;

[0161] The latent representation at time step s is x s ;

[0162] Based on the initial value x s Obtain the latent representation x at time step t t Time t∈[0,s]:

[0163]

[0164] Where, λ t It is a strictly decreasing function of t;

[0165] σ t It is the coefficient related to the noise at time step t, σ s It is the coefficient related to the time step s and the noise;

[0166] f θ It is the output of the diffusion model at time step t;

[0167] Let λ represent the potential vector in the soft absorption state, and let λ represent the time-dependent variable used to simplify the noise control process.

[0168] λ t Let λ represent the variable with respect to time step t used to simplify the noise control process.s This represents a variable about time step s used to simplify the noise control process;

[0169] σ t Relative to β t It is monotonous, and f θ Used to transfer corrupted data x t Restore the data prediction model to x0;

[0170] In addition, this formula requires Approximation. The integral can be analytically computed by repeatedly applying the block integral n times, and only the first few terms can be approximated, while higher-order error terms are ignored. In our experiments, we use second-order integrals. The algorithm remains applicable in our method after discretization and denoising because... Consistent with the training objectives.

[0171] Steps 2 and 3: Let time step t = t + 1; repeat steps 2 and 2 until the overall objective function converges, and obtain the trained diffusion model.

[0172] Step 24: Transfer the sentiment-labeled training set D from the source domain... s and the generated original target domain guiding sample set D r The trained diffusion model is input as sample pairs, and the trained diffusion model generates a qualified counterfactual target domain sample set D. c .

[0173] Based on this, using the source domain dataset D s and the corresponding generated counterfactual target domain dataset D c Train the classifier, where D c The sample labels in the D are paired and correspond to each other. s and D r The labels in the target domain are consistent with those in the test samples to predict the labels.

[0174] Training and sampling are stopped at 2000 time steps to obtain D. c Source domain dataset D s and the corresponding generated counterfactual target domain dataset D c Input the classifier and train it until convergence to obtain a trained classifier.

[0175] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0176] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One through Eight in that the target domain sample data classification layer is used to train a BERT classification model using samples labeled with sentiment and samples labeled with domain, respectively. The trained BERT classification model is then used to classify the unlabeled sentiment label test set D of the target domain. t Sentiment categorization includes the following steps:

[0177] Step 31: Obtain the trained sentiment BERT classification model; the specific process is as follows:

[0178] The trained diffusion model generates a qualified counterfactual target domain sample set D. c The training set D with sentiment labels in the source domain s Input the BERT classification model, train the BERT classification model, and obtain a trained sentiment BERT classification model;

[0179] Step 3.2: Test set D of the target domain without labeled sentiment tags t Input a pre-trained sentiment BERT classification model, and the pre-trained sentiment BERT classification model outputs unlabeled sentiment samples w from the test set in the target domain. t The sentiment classification results.

[0180] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.

[0181] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement, characterized by: The specific process of the method is as follows: Step 1: Obtain sample data from the sentiment dataset; The sample data in the sentiment dataset are all labeled with sentiment. The sample data within the sentiment dataset carries different types of sentiment labels; The sample data in the sentiment dataset is divided into a training set and a test set; The training and test sets come from different domains; The sentiment tags on the test set are not visible to the model; The training and test sets include samples w and sentiment s; The sample data is text data; The emotional label is either positive or negative; The area in question is either the kitchen or books; Step 2: Establish CDA 2 Model; CDA 2 The model includes: a raw target domain sample generation layer, a counterfactual target domain sample generation layer, and a target domain sample data classification layer; The original target domain sample generation layer, under a masking mechanism, generates samples from the sentiment-labeled training set in the source domain through domain destruction and reconstruction. s And the unlabeled sentiment tags in the test set of the target domain w t Generate original target domain guide samples w r ; The counterfactual target domain sample generation layer is used to generate samples from the source domain's sentiment-labeled training set D. s and the generated original target domain guiding sample set D r The sample pairs are input into the diffusion model, which generates a qualified counterfactual target domain sample set D. C ; The test set D of the target domain without labeled sentiment tags t Samples w from the test set of unlabeled sentiment data in the target domain t composition; The counterfactual target domain sample set D C Sample w from the counterfactual target domain sample set c composition; The original target domain guiding sample set D r Sample w is guided by the original target domain in the sample set. r composition; The target domain sample data classification layer is used to train the BERT classification model using samples labeled with sentiment and samples labeled with domain, respectively. The trained BERT classification model is then used to test the unlabeled sentiment data set D in the target domain. t Perform sentiment categorization.

2. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 1, characterized in that: The original target domain sample generation layer, under a masking mechanism, uses domain destruction and domain reconstruction to generate samples w from the sentiment-labeled training set of the source domain. s And the unlabeled sentiment tags in the test set of the target domain w t Generate original target domain guide samples w r ; The specific process is as follows: Step 11, Domain Destruction: This involves modifying the sentiment-labeled training set D from the source domain. s By masking specific domain terms, a sentiment-labeled training set D from the source domain is obtained. s The specific process is as follows: 1) Training set D s The document includes sentence samples, and each sentence contains words; Let w = {w1, w2, ..., wn} j ,…,w n } represents the sentiment-labeled training set D from the source domain. s A sample of sentences of length n, w j This represents a word in a sentence sample; 2) The training set D with sentiment labels in the source domain s Mask all unary words in the text; 3) The training set D with sentiment labels in the source domain s Mask all binary words in the string; 4) The training set D with sentiment labels in the source domain s Mask all triples in the source domain to obtain the masked training set D with sentiment labels. s ; Steps 1 and 2: Domain Reconstruction: Based on unlabeled sentiment tags in the test set w of the target domain t and the sentiment-labeled training set D of the masked source domain s The original target domain guided sample w is obtained. r .

3. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 2, characterized in that: In section 2), the training set D with sentiment labels in the source domain... s Mask all unary words in the text; the specific process is as follows: Calculate the masking score M(w) for each unary word. j D s D t If the masking score of a unary word is greater than the threshold τ, then the unary word is masked; if the masking score of a unary word is less than or equal to the threshold τ, then the unary word is not masked.

4. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 3, characterized in that: In point 3), the training set D with sentiment labels in the source domain... s Mask all bigrams in the string; the specific process is as follows: For sentence samples w = {w1, w1, ..., w j ,…,w n For any two adjacent bigrams in a given bigram, if neither of the two unary words in the bigram is masked, then calculate the masking score M(w) for each unary word. j D s D t If the sum of the masking scores of two unary words is greater than the threshold τ, then the two unary words are masked; if the sum of the masking scores of two unary words is less than or equal to the threshold τ, then the two unary words are not masked. If one or both unary words in a binary word are masked, then the binary word is not masked.

5. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 4, characterized in that: In section 4), the training set D with sentiment labels in the source domain... s Mask all trigrams in the string; the specific process is as follows: For sentence samples w = {w1, w1, ..., w j ,…,w n For any three adjacent triplets in the unary, if none of the three unary words in the triplet are masked, then calculate the masking score M(w) for each unary word. j D s D t If the sum of the masking scores of the three unary words is greater than the threshold τ, then the three unary words are masked; if the sum of the masking scores of the three unary words is less than or equal to the threshold τ, then the three unary words are not masked. If one, two, or all three unary words in a ternary word are masked, then the ternary word is not masked.

6. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 5, characterized in that: The masking fraction M(w) j D s D t The calculation process for ) is as follows: 1) Calculate the word w in the sentence sample j Belonging to the source domain sample set D s The probability P(D) s |w j The expression is: in, D represents the source domain sample set. s The total number of samples included; D represents the source domain sample set. s The number of samples containing domain terms; α represents the smoothing hyperparameter, and ∝ represents the correlation; 2) Calculate the word w in the sentence sample j Belonging to the target domain sample set D t The probability P(D) t |w j The expression is: in, D represents the target domain sample set. t The total number of samples included; D represents the target domain sample set. t A sample of domain terms included; α represents the smoothing hyperparameter, and ∝ represents the correlation; 3) Based on P(D) s |w j ) Calculate the entropy H(D) s |w j ); 4) Based on P(D) t |w j ) Calculate the entropy H(D) t |w j ); 5) Based on P(D) s |w j ) and H(D s |w j ), calculate the word w in the sentence sample j and source domain sample set D s The correlation between ρ(w) j D s The expression is: Where N represents the number of unlabeled domains; 6) Based on P(D) t |w j ) and H(D t |w j ), calculate the word w in the sentence sample j and the target domain sample set D t The correlation between ρ(w) j D t The expression is: Where N represents the number of unlabeled domains; 7) Based on the word w in the sample j and source domain sample set D s The correlation between ρ(w) j D s ), and the word w in the sample j and the target domain sample set D t The correlation between ρ(w) j D t ), calculate the word w in the sample j In the source domain sample set D s and the target domain sample set D t The masking score M(w) j D s D t ): M(w j ,D s ,D t )=ρ(w j ,D s )-ρ(w j ,D t ) ρ(w j D s ) represents the word w in the sample j and source domain sample set D s The relationship between them; ρ(w j D t ) represents the word w in the sample j and the target domain sample set D t The relationship between them.

7. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 6, characterized in that: The domain reconstruction in steps one and two involves using samples w from the test set of unlabeled sentiment tags in the target domain. t and the sentiment-labeled training set D of the masked source domain s The original target domain guided sample w is obtained. r ; The specific process is as follows: Step 1, Step 2, Step 1 The masked source domain training set D obtained step by step is used to... s denoted as M(w) r ); The test set D, which is not labeled with sentiment in the target domain. t The domain direction vector v t With M(w) r The embedding vector v r Concatenate matrices by splicing along the feature dimensions; The specific process is as follows: 1) Design a learnable embedding matrix containing K×V size embedding vectors, where the K×V size embedding vectors are used as M(w r The embedding vector v r K represents the number of vectors, and V represents the dimension of the vectors. 2) Initialize the domain direction vector using the domain name and domain terms, which serves as the test set D for the target domain without labeled sentiment. t The domain direction vector v t ; 3) The test set D, which is not labeled with sentiment tags in the target domain. t The domain direction vector v t With M(w) r The embedding vector v r Concatenate matrices by splicing along the feature dimensions; Step 122: Input the concatenated matrix into the T5 generative model based on the encoder-decoder architecture. The T5 generative model generates the original target domain guide sample w. r ; Each original target domain guides samples w r Corresponding to a single sample w s w r and w s They share the same emotional tags.

8. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 7, characterized in that: The counterfactual target domain sample generation layer is used to generate samples from the source domain training set D with sentiment labels. s and the generated original target domain guiding sample set D r The sample pairs are input into the diffusion model, which generates a qualified counterfactual target domain sample set D. c The specific process is as follows: Step 2.1: Training phase of the diffusion model; the specific process is as follows: 1) Within the diffusion model, the samples w in the training set with sentiment labels from the source domain are... s and the original target domain guided sample set sample w r Connect them to get the sample 2) Set the time step t = 0; 3) Using x in the diffusion model t express The potential representation of; The specific process is as follows: Sample Embedded into a continuous feature space, the forward process of the diffusion model is obtained; Where I is an identity matrix; N() represents a normal distribution; β0 represents the scheduling coefficient of the noise variance. Indicates the sample Embedded; x t This represents the perturbed data after the forward noise addition process; This indicates that the training set samples w with sentiment labels from the source domain will be used. s and the original target domain guided sample set sample w r The connected samples; This represents the forward process of the diffusion model; 4) Based on x t Derive the latent representation x at time step t t as follows: in, denoted by , representing the noise attenuation coefficient at time step t in the diffusion model; ∈ represents normally distributed noise; 5) The latent representation x of time step t t The i-th token in the process is replaced with a soft-absorbing state n; The soft-absorbing state n has the same dimension as the token; in, This represents the i-th token at time step t after the replacement; The latent representation x at time step t t The i-th token in the array; η represents the flag used to determine whether to perform a soft absorption state substitution; 6) The overall objective function is derived by adding the two components together: in, These are weight parameters; This is the loss function for the diffusion model; MMD loss function; The MMD loss function Represented as Where, N C This represents the total number of samples in the counterfactual target domain sample set; N t This represents the total number of samples in the test set that are not labeled with sentiment tags in the target domain; k() represents the Gaussian kernel function; Sentence and sentences The squared distance between them; Step 22: Sampling Stage of the Diffusion Model. The specific process is as follows: Set the initial value of the time step to s, s>0; The latent representation at time step s is x s ; Based on the initial value x s Obtain the latent representation x at time step t t Time t∈[0,s]: Where, σ t It is the coefficient related to the noise at time step t, σ s It is the coefficient related to the time step s and the noise; f θ It is the output of the diffusion model at time step t; Let λ represent the potential vector in the soft absorption state, and let λ represent the variable with respect to time. λ t Let λ be a variable representing time step t. s Represents a variable related to time step s; Steps 2 and 3: Let time step t = t + 1; repeat steps 2 and 2 until the overall objective function converges, and obtain the trained diffusion model. Step 24: Transfer the sentiment-labeled training set D from the source domain... s and the generated original target domain guiding sample set D r The trained diffusion model is input as sample pairs, and the trained diffusion model generates a qualified counterfactual target domain sample set D. c .

9. The sentiment analysis method based on low-resource cross-domain diffusion counterfactual data enhancement according to claim 8, characterized in that: The target domain sample data classification layer is used to train a BERT classification model using samples labeled with sentiment and samples labeled with domain, respectively. The trained BERT classification model is then used to test the unlabeled sentiment set D of the target domain. t Sentiment categorization includes the following steps: Step 31: Obtain the trained sentiment BERT classification model; the specific process is as follows: The trained diffusion model generates a qualified counterfactual target domain sample set D. c The training set D with sentiment labels in the source domain s Input the BERT classification model, train the BERT classification model, and obtain a trained sentiment BERT classification model; Step 3.2: Test set D of the target domain without labeled sentiment tags t Input a pre-trained sentiment BERT classification model, and the pre-trained sentiment BERT classification model outputs unlabeled sentiment samples w from the test set in the target domain. t The sentiment classification results.

Images