A cross-domain sentiment classification method based on contrastive learning

Abstract

The present application belongs to the field of big data analysis, and particularly relates to a cross-domain sentiment classification method based on contrast learning. The method is used for enhancing the feature extraction of a sentiment classification model on text information and the analysis ability of the cross-domain sentiment, and comprises the following steps: S1: constructing a cross-domain sentiment training model comprising a source domain feature extractor, a target domain feature extractor, a sentiment classifier, a domain aligner and a contrast learner; S2: preprocessing review corpus to form a required review dataset; S3: designing an overall target loss function required for model training; S4: training the cross-domain sentiment training model using an Adam optimizer; S5: constructing a required cross-domain sentiment classification network using the target domain feature extractor and the sentiment classifier of the best cross-domain sentiment training model; and S6: performing cross-domain sentiment classification on target text information using the cross-domain sentiment classification network. The present application solves the problem of insufficient sentiment classification precision caused by the neglect of domain-specific features of the target domain in the prior art.

Description

Technical Field

[0001] This invention belongs to the field of big data analysis, specifically relating to a cross-domain sentiment classification method based on contrastive learning. Background Technology

[0002] With the continuous development of computer technology, virtual social networks are becoming an indispensable part of people's lives. In the era of information explosion, online comment text is increasing, and sentiment classification technology is playing an increasingly important role in automatically determining the sentiment polarity of text. However, text sentiment classification technology requires a large number of labeled samples. The existing large amount of comment data that can be collected from social networks is blank data lacking accurate sentiment labels, which brings difficulties to the training of network models related to cross-domain sentiment classification.

[0003] In industry, it's common to find source domain data that is abundant and labeled, while target domain data lacks labels. Therefore, some researchers have proposed cross-domain sentiment classification techniques. These techniques aim to improve sentiment classification performance in the target domain by using labeled source domain data and unlabeled target domain data. Domain shift between the source and target domains is a major challenge in cross-domain sentiment classification, primarily stemming from distributional differences between the two domains. For example, words used in the book domain differ significantly from those used in the restaurant domain. Currently, unsupervised domain adaptation (UDA) techniques are used to address this problem. Essentially, UDA utilizes unlabeled data in the target domain to minimize domain shift by aligning features between the source and target domains.

[0004] Current research on cross-domain sentiment classification can be broadly categorized into two types: The first type comprises task-independent methods, including divergence minimization, instance reweighting, and domain adversarial training. Some studies also combine task-independent methods with natural language processing-specific methods or models. Divergence minimization methods achieve domain-invariant representations by minimizing the data distributions of the source and target domains. Instance-weighted cross-domain sentiment classification reduces the distributional differences between the source and target domains by assigning higher weights to samples similar to the target domain during training. However, this method inevitably introduces negative transfer problems during the transfer process, potentially degrading the target domain sentiment classification performance. Domain adversarial training methods extract domain-invariant features by minimizing the classification loss function of the source samples and the domain confusion loss function of all samples. The second type is pivot-based methods. These methods learn domain-invariant features by leveraging the correlation between pivots and non-pivots to compensate for domain differences. To date, the most widely used pivot-based method is Structural Correspondence Learning (SCL) and its variants. In SCL, the pivot is generally defined as a word that frequently appears in both the source and target domains. The model can effectively learn the domain-invariant features of axes, but it is more challenging for non-axis features because they have domain-specific meanings.

[0005] Both of the aforementioned conventional approaches can effectively extract domain-invariant features from the source and target domains, but they neglect domain-specific features of the target domain. Consequently, the sentiment classification accuracy of the network model is relatively low, especially when there are significant differences between the source and target domain data, the accuracy of the trained network model often falls short of expectations. Summary of the Invention

[0006] To address the problem that existing sentiment classification techniques can only extract domain-invariant features of the source and target domains while ignoring domain-specific features of the target domain, this invention provides a cross-domain sentiment classification method based on contrastive learning.

[0007] This invention is achieved using the following technical solution:

[0008] A cross-domain sentiment classification method based on contrastive learning is proposed to enhance the ability of sentiment classification models to extract features from text information and analyze cross-domain sentiment. This cross-domain sentiment classification method includes the following steps:

[0009] S1: Construct a cross-domain sentiment training model that includes a source domain feature extractor, a target domain feature extractor, a sentiment classifier, a domain aligner, and a contrastive learner. The data processing method for the cross-domain sentiment training model is as follows:

[0010] (1) Use the source domain feature extractor to extract features from the source domain data Ds to obtain the source domain features Fs.

[0011] (2) Perform data augmentation on the target domain data Dt to obtain augmented data Dt′.

[0012] (3) Use the target domain feature extractor, which shares parameters with the source domain feature extractor, to extract features from the target domain data Dt and the enhanced data Dt′, and obtain the target domain feature Ft and the enhanced feature Ft′ respectively.

[0013] (4) The output of the source domain feature Fs is divided into two paths. One path is input into the sentiment classifier, and the corresponding sentiment polarity classification result is obtained after processing by the sentiment classifier; the other path is input into the domain aligner.

[0014] (5) The target domain feature Ft extracted by the target domain feature extractor is divided into two paths. One path is input to the domain aligner, which calculates the difference between the two domains based on the source domain feature Fs and the target domain feature Ft. The other path is input to the contrast learner.

[0015] (6) The enhanced feature Ft′ extracted by the target domain feature extractor is input into the contrastive learner, which combines the target domain feature Ft and the enhanced feature Ft′ to extract specific features in the target domain.

[0016] S2: Obtain a large corpus of comment data and preprocess it to form the required comment dataset. The comment dataset includes source domain data Ds and target domain data Dt. The source domain data Ds contains sentiment labels, while the target domain data Dt does not contain labels.

[0017] S3: Design the overall objective loss function L required for training the cross-domain emotion training model; including the following steps:

[0018] S31: Based on the classification objective, the cross-entropy loss L... sent Source domain sentiment classification loss as a sentiment classifier.

[0019] S32: Based on the learning characteristics of discriminative features, the objective function for evaluating the performance of the contrastive learner in extracting specific features of the target domain is set as the contrastive loss L. con .

[0020] S33: Perform feature alignment on the source and target domain data to extract domain-invariant features, and then set the loss function of the domain aligner to the alignment difference loss L. align .

[0021] S34: Minimize the cross-entropy loss L sent , contrast loss L con Alignment difference loss L align As a cross-domain emotion

[0022] The overall objective loss function L during the training phase of the model is:

[0023] L = L sent +αL con +βL align

[0024] Where α and β are the weighting factors for the contrast loss term and the alignment difference loss term, respectively.

[0025] S4: Train the cross-domain sentiment training model using the Adam optimizer. The training phase includes the following:

[0026] S41: Use the target domain data Dt and part of the source domain data Ds in the comment dataset as the training set;

[0027] The remaining data in the source domain Ds is used as the validation set.

[0028] S42: Use the overall objective loss function L designed as the optimization function during the training phase.

[0029] S43: Set the parameters for the training phase, including: learning rate, number of iterations, batch size, and hyperparameters in the overall objective loss function L.

[0030] S44: Use the training set to perform batch iterative training on the cross-domain emotion training model; and save the model parameters after each round of training.

[0031] S45: Using accuracy as the evaluation metric, after each training round, validate the trained network model using a validation set and calculate the corresponding evaluation metric. Update and save the model parameters of the cross-domain sentiment training model with the best evaluation metric.

[0032] S5: Select the cross-domain sentiment training model with the best evaluation metric after training, use the target domain feature extractor as the feature extraction part, and use the sentiment classifier as the classification part to obtain the required cross-domain sentiment classification network.

[0033] S6: The cross-domain sentiment classification network is used to extract and analyze the features of the input target text information in sequence, and then output the corresponding cross-domain sentiment classification results.

[0034] As a further improvement of the present invention, in the cross-domain sentiment training model constructed in step S1, both the source domain feature extractor and the target domain feature extractor use the pre-trained language model BERT as the backbone to encode the contextual information of the comment statement; the two feature extractors share parameters.

[0035] In the cross-domain sentiment training model constructed in step S1, the sentiment classifier consists of a multilayer perceptron (MLP) and a softmax layer. The MLP comprises four layers: a fully connected layer, a ReLU activation function layer, a dropout layer, and another fully connected layer. The output representation from the MLP is fed into the softmax layer, and the corresponding loss is calculated from there.

[0036] Domain alignment consists of a multilayer perceptron (MLP) and a contrastive learning layer, with the CMD layer consisting of multiple fully connected layers; the contrastive learner consists of a multilayer perceptron (MLP) and a contrastive learning layer.

[0037] As a further improvement of the present invention, in step S1, back-translation is used to perform data augmentation on each comment text data in the target domain data Dt, thereby obtaining positive sample pairs for each comment text data.

[0038] As a further improvement of the present invention, the preprocessing of the comment corpus in step S2 is as follows:

[0039] First, load the comment corpus and a pre-trained language model. The pre-trained language model chosen is either BERT or RoBERTa. Then, use the pre-trained language model to preprocess the comment corpus and format the text data.

[0040] As a further improvement of this invention, the steps of text preprocessing and text data formatting for the pre-trained language model are as follows:

[0041] S001: Use the nltk tokenizer to segment the comment statement and separate the segmented tokens with spaces.

[0042] S002: Add two special tokens, [CLS] and [SEP], after the token sequence of the comment statement to construct a general input form: S = {[CLS], w1, w2, ..., w...} n , [SEP]}.

[0043] Where n represents the total number of tokens in the comment statement, w n This indicates the nth token in the comment statement; [CLS] tokens are used for categorization, and [SEP] tokens are used to separate two sentences.

[0044] S003: Pad the token sequence of each comment statement to make it 256 in length. Statements longer than 256 are truncated, and statements shorter than 256 are padded with zeros.

[0045] S004: Use the tokenizer of the pre-trained language model to perform a tokenization operation on each token word in the comment statement; obtain the source domain data Ds or target domain data Dt in the required batch data form.

[0046] As a further improvement of the present invention, in step S31, the cross-entropy loss L of the sentiment classifier sent The function is as follows:

[0047]

[0048] Where, n s y represents the number of data samples in the original domain. i Let y be the label of the i-th sample in the original domain data. i i represents the predicted label of the model on the i-th sample in the original domain data.

[0049] As a further improvement of the present invention, in step S32, the contrastive loss L of the contrast learner con as follows:

[0050]

[0051] Where k represents the index of each term in the summation function; z i ,z j sim(z) represents a pair of positive samples; i , z j ) represents z i and z j The cosine similarity; N is the batch size; t is the temperature hyperparameter, which focuses the model update on difficult negative examples and penalizes them accordingly; I is the equality indicator function, which is zero when k = i and one otherwise.

[0052] As a further improvement of the present invention, in step S33, the alignment difference loss L align The function is as follows:

[0053]

[0054] Where N represents the number of fully connected layers; and W represents the features of the source and target domains after passing through the i-th fully connected layer, respectively. i S and W i t These represent the weights of the i-th fully connected layer in the source and target domains, respectively. and These represent the biases of the i-th fully connected layer in the source and target domains, respectively; F represents the ReLU activation function. This is expressed as a K-order central moment calculation function; specifically as follows:

[0055]

[0056] Where A and B are two bounded random variables with a range of [a, b]; E(A) and E(B) are the expected values ​​calculated for samples A and B, respectively; C k (A) and C k (B) represents the k-th order central moment, which is the expected value of the distance of the sample from the center to the power of k.

[0057] As a further improvement of the present invention, in the parameters set in step S43 for the training phase, the learning rate is 2.0e-5, and the contrastive loss L in the contrastive learner is... con The temperature parameter is 0.065; the batch size is set to 128; and the number of training iterations is 32.

[0058] The technical solution provided by this invention has the following beneficial effects:

[0059] This invention provides a cross-domain sentiment classification method based on contrastive learning. The method first constructs two Siamese networks with shared weights for the domain feature extraction network: a source domain feature extractor and a target domain feature extractor. Then, a cross-domain sentiment training model is built using the source domain feature extractor, target domain feature extractor, sentiment classifier, domain aligner, and contrastive learner. Next, the cross-domain sentiment training model is trained using three objectives extracted in this invention. The first objective is to minimize the sentiment classification loss in the source domain; the second is to minimize the contrastive loss in the target domain and extract discriminative features in the target domain.

[0060] The third objective is to minimize the alignment loss between the source and target domains and extract domain-invariant features. Finally, the target domain feature extractor and sentiment classifier in the trained network model are used to construct the required cross-domain sentiment classification method to classify the sentiment polarity of the target user. The novel cross-domain sentiment classification method provided by this invention can extract both domain-invariant features from the two domains and domain-specific features from the target domain, thereby significantly improving the network's classification performance in the target domain and increasing the sentiment classification accuracy of the network model. Attached Figure Description

[0061] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0062] Figure 1 This is the complete process of the cross-domain sentiment classification method based on contrastive learning provided in Embodiment 1 of the present invention.

[0063] Figure 2 This is the learning framework for the cross-domain emotion training model constructed in Embodiment 1 of the present invention.

[0064] Figure 3 These are the text preprocessing and text data formatting steps for the pre-trained language model in Embodiment 1 of the present invention.

[0065] Figure 4 This is a network architecture diagram of a cross-domain sentiment classification network model provided in Embodiment 1 of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0067] Example 1

[0068] This embodiment provides a cross-domain sentiment classification method based on contrastive learning, which enhances the ability of sentiment classification models to extract features from textual information and analyze cross-domain sentiment. It addresses the problem that traditional sentiment classification models only focus on extracting domain-invariant features from the source and target domains, neglecting domain-specific features of the target domain; thus, the model identifies sentiment polarity more accurately. The core strategy of this cross-domain sentiment classification method based on contrastive learning is to build a contrastive learning framework using a feature extraction network and a sentiment classifier, and then adjust the model parameters of the network model according to three different optimization objectives, thereby improving the model's accuracy in sentiment polarity classification.

[0069] Contrastive learning is a self-supervised learning method that does not rely on labeled data. It autonomously learns knowledge from large amounts of data and encodes it into high-level representations. Therefore, this embodiment utilizes the constructed contrastive learning framework to enable the feature extraction module to learn domain-specific features of the target domain, thereby improving the network's ability to extract and analyze cross-domain information; ultimately improving the accuracy of cross-domain sentiment classification results.

[0070] Specifically, the complete process of the cross-domain sentiment classification method based on contrastive learning provided in this embodiment is as follows: Figure 1 As shown, it includes the following steps:

[0071] S1: Construct a cross-domain sentiment training model comprising a source domain feature extractor, a target domain feature extractor, a sentiment classifier, a domain aligner, and a contrastive learner. Both the source and target domain feature extractors use the pre-trained language model BERT as their backbone to encode the contextual information of the comment statements; the two feature extractors share parameters. The sentiment classifier consists of a multilayer perceptron (MLP) and a softmax layer. The MLP comprises four layers: a fully connected layer, a ReLU activation function layer, a dropout layer, and a fully connected layer. The output representation after passing through the MLP is fed into the softmax layer, and the corresponding loss is calculated from this layer. The domain aligner consists of an MLP and a contrastive learning layer, with the CMD layer consisting of multiple fully connected layers. The contrastive learner consists of an MLP and a contrastive learning layer.

[0072] The learning framework of the cross-domain emotion training model constructed in this embodiment is as follows: Figure 2 As shown, combined with Figure 2 The data processing method for cross-domain sentiment training models is as follows:

[0073] (1) Use the source domain feature extractor to extract features from the source domain data Ds to obtain the source domain features Fs.

[0074] (2) Perform data augmentation on the target domain data Dt to obtain augmented data Dt′. In the specific implementation process, the back-translation method is used to augment the data of each comment text in the target domain data Dt, thereby obtaining positive sample pairs of each comment text data.

[0075] (3) Use the target domain feature extractor, which shares parameters with the source domain feature extractor, to extract features from the target domain data Dt and the enhanced data Dt′, and obtain the target domain feature Ft and the enhanced feature Ft′ respectively.

[0076] (4) The output of the source domain feature Fs is divided into two paths. One path is input into the sentiment classifier, and the corresponding sentiment polarity classification result is obtained after processing by the sentiment classifier; the other path is input into the domain aligner.

[0077] (5) The target domain feature Ft extracted by the target domain feature extractor is divided into two paths. One path is input to the domain aligner, which calculates the difference between the two domains based on the source domain feature Fs and the target domain feature Ft. The other path is input to the contrast learner.

[0078] (6) The enhanced feature Ft′ extracted by the target domain feature extractor is input into the contrastive learner, which combines the target domain feature Ft and the enhanced feature Ft′ to extract specific features in the target domain.

[0079] S2: Obtain a large corpus of comment data and preprocess it to form the required comment dataset. The comment dataset includes source domain data Ds and target domain data Dt. The source domain data Ds contains sentiment labels, while the target domain data Dt does not contain labels.

[0080] The preprocessing of the comment corpus is as follows: First, load the comment corpus and a pre-trained language model. The pre-trained language model is either BERT or RoBERTa. Then, the pre-trained language model is used to preprocess the comment corpus and format the text data.

[0081] In detail, such as Figure 3 As shown, the steps for text preprocessing and text data formatting in a pre-trained language model are as follows:

[0082] S001: Use the nltk tokenizer to segment the comment statement and separate the segmented tokens with spaces.

[0083] S002: Add two special tokens, [CLS] and [SEP], after the token sequence of the comment statement to construct a general input form: S = {[CLS], w1, w2, ..., w...} n , [SEP]}.

[0084] Where n represents the total number of tokens in the comment statement, w n This indicates the nth token in the comment statement; [CLS] tokens are used for categorization, and [SEP] tokens are used to separate two sentences.

[0085] S003: Pad the token sequence of each comment statement to make it 256 in length. Statements longer than 256 are truncated, and statements shorter than 256 are padded with zeros.

[0086] S004: Use the tokenizer of the pre-trained language model to perform a tokenization operation on each token word in the comment statement; obtain the source domain data Ds or target domain data Dt in the required batch data form.

[0087] S3: Design the overall objective loss function L required for training the cross-domain emotion training model; including the following steps:

[0088] S31: Based on the classification objective, the cross-entropy loss L... sent Source domain sentiment classification loss as a sentiment classifier.

[0089] Cross-entropy loss L of sentiment classifier sent The function is as follows:

[0090]

[0091] Where, n s y represents the number of data samples in the original domain. i Let y be the label of the i-th sample in the original domain data. i ′ represents the predicted label of the model on the i-th sample in the original domain data.

[0092] S32: Based on the learning characteristics of discriminative features, the objective function for evaluating the performance of the contrastive learner in extracting specific features of the target domain is set as the contrastive loss L. con .

[0093] In the learning framework constructed in step S1, the vector f obtained after passing the original data and augmented data through the feature extractor is... t and f t The input to the MLP helps in learning discriminative features; then the output vector z... i and z j Find the contrastive loss. What is the contrastive loss L of the contrastive learner? con as follows:

[0094]

[0095] Where k represents the index of each term in the summation function; z i ,z j sim(z) represents a pair of positive samples; i , z j ) represents z i and z j The cosine similarity; N is the batch size; t is the temperature hyperparameter, which focuses the model update on difficult negative examples and penalizes them accordingly; I is the equality indicator function, which is zero when k = i and one otherwise.

[0096] S33: Perform feature alignment on the source and target domain data to extract domain-invariant features, and then set the loss function of the domain aligner to the alignment difference loss L. align .

[0097] Domain adaptation is an effective method that aims to capture domain-invariant features by minimizing the distance between two domains. Our method uses central moment difference to estimate domain differences and constructs a domain alignment loss at a high-level semantic layer. By minimizing this loss term, the distributional difference between the two domains is reduced, thereby extracting domain-invariant features. Specifically, in this embodiment, the alignment difference loss is calculated from the vectors obtained after the source and target domain data have passed through the feature extractor. The alignment difference loss L... align The function is as follows:

[0098]

[0099] Where N represents the number of fully connected layers; and W represents the features of the source and target domains after passing through the i-th fully connected layer, respectively. i S and W i t These represent the weights of the i-th fully connected layer in the source and target domains, respectively. and These represent the biases of the i-th fully connected layer in the source and target domains, respectively; F represents the ReLU activation function. This is expressed as a K-order central moment calculation function; specifically as follows:

[0100]

[0101] Where A and B are two bounded random variables with a range of [a, b]; E(A) and E(B) are the expected values ​​calculated for samples A and B, respectively; C k (A) and C k (B) represents the k-th order central moment, which is the expected value of the distance of the sample from the center to the power of k.

[0102] S34: Minimize the cross-entropy loss L sent , contrast loss L con Alignment difference loss L align The overall objective loss function L during the training phase of the cross-domain emotion training model is:

[0103] L = L sent +αL con +βL align

[0104] Where α and β are the weighting factors for the contrast loss term and the alignment difference loss term, respectively.

[0105] S4: Train the cross-domain sentiment training model using the Adam optimizer. The training phase includes the following:

[0106] S41: Use the target domain data Dt and part of the source domain data Ds in the review dataset as the training set; use the remaining source domain data Ds as the validation set.

[0107] S42: Use the overall objective loss function L designed as the optimization function during the training phase.

[0108] S43: Set the parameters for the training phase, including: learning rate, number of iterations, batch size, and hyperparameters in the overall objective loss function L. In this embodiment, the learning rate is set to 2.0e-5, and the contrastive loss L in the contrastive learner is...con The temperature parameter is 0.065; the batch size is set to 128; and the number of training iterations is 32.

[0109] S44: Use the training set to perform batch iterative training on the cross-domain emotion training model; and save the model parameters after each round of training.

[0110] S45: Using accuracy as the evaluation metric, after each training round, validate the trained network model using a validation set and calculate the corresponding evaluation metric. Update and save the model parameters of the cross-domain sentiment training model with the best evaluation metric.

[0111] S5: Select the cross-domain sentiment training model with the best evaluation metric after training, use the target domain feature extractor as the feature extraction part, and use the sentiment classifier as the classification part to obtain the required cross-domain sentiment classification network.

[0112] S6: The cross-domain sentiment classification network is used to extract and analyze the features of the input target text information in sequence, and then output the corresponding cross-domain sentiment classification results.

[0113] The cross-domain sentiment classification method based on contrastive learning provided in this embodiment first constructs two Siamese networks with shared weights for the domain feature extraction network: a source domain feature extractor and a target domain feature extractor. Then, a cross-domain sentiment training model is built using the source domain feature extractor, target domain feature extractor, sentiment classifier, domain aligner, and contrastive learner. Next, the cross-domain sentiment training model is trained using three objectives extracted in this invention. The first objective is to minimize the sentiment classification loss in the source domain; the second is to minimize the contrastive loss in the target domain, extracting discriminative features in the target domain; and the third is to minimize the alignment loss between the source and target domains, extracting domain-invariant features. Finally, the target domain feature extractor and sentiment classifier in the trained network model are used to construct the required cross-domain sentiment classification method to classify the sentiment polarity of target users. The novel cross-domain sentiment classification method provided by this invention can extract both domain-invariant features from the two domains and domain-specific features from the target domain, thereby significantly improving the network's classification performance in the target domain and increasing the sentiment classification accuracy of the network model.

[0114] Example 2

[0115] Building upon Example 1, this example further provides a cross-domain sentiment classification network model, the framework of which is as follows: Figure 4As shown, the system includes a preprocessing module, a domain feature extraction module, and a sentiment classifier. The domain feature extraction module uses a pre-trained language model BERT as its backbone to encode the contextual information of the comment statements. The sentiment classifier consists of a multilayer perceptron (MLP) and softmax layers. The MLP comprises four layers: a fully connected layer, a ReLU activation function layer, a dropout layer, and a fully connected layer. Specifically, the model parameters for both the domain feature extraction module and the sentiment classifier are the same as those used in Example 1, trained using the contrastive learning framework.

[0116] The preprocessing module is used to preprocess the comment corpus, thereby forming formatted text data. Specifically, the text formatting process of the comment corpus is as follows: First, load the comment corpus and the pre-trained language model. The BERT model is selected as the pre-trained language model. Then, the comment corpus is preprocessed and formatted using the pre-trained language model: the steps are as follows: (1) Use the nltk tokenizer to segment the comment statements and separate the segmented tokens with spaces. (2) Add two special tokens after the segmented token sequence of the comment statements: [CLS], [SEP], thereby constructing a general input form: S = {[CLS], w1, w2, w...} n ,[SEP]}。 Where n represents the total number of token words in the comment statement, w n This represents the nth token in the comment statement; [CLS] tokens are used for categorization.

[0117] [SEP] token words are used to separate two sentences. (3) Pad the token word sequence of each comment sentence to make its length 256. Sentences longer than 256 are truncated, and sentences shorter than 256 are padded with zeros. (4) Use the tokenizer of the pre-trained language model to tokenize each token word in the comment sentence.

[0118] The cross-domain sentiment classification network model provided in this embodiment first converts the comment corpus to be classified into standard formatted text data through a preprocessing module. Then, the formatted text data is input into a domain feature extraction module, where the domain feature extractor extracts multi-dimensional features and contextual information contained in the text information. The extracted features are input into a sentiment classifier, and the sentiment classification outputs the classification result of the sentiment polarity corresponding to the current comment corpus.

[0119] Example 3

[0120] Building upon Example 1, this embodiment further provides a cross-domain sentiment classification device. This device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The functional modules within the cross-domain sentiment classification device are deployed using the cross-domain sentiment classification network model described in Example 2. When the processor executes the computer program, it implements the steps of the contrastive learning-based cross-domain sentiment classification method described in Example 1, and then outputs a classification result of the corresponding user's sentiment polarity based on the input comment corpus.

[0121] The computer device can be a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers), etc., capable of executing programs. The computer device in this embodiment includes, but is not limited to, a memory and a processor that can be interconnected via a system bus.

[0122] In this embodiment, the memory (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of a computer device, such as the hard disk or RAM of the computer device. In other embodiments, the memory can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device. Of course, the memory can also include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is typically used to store the operating system and various application software installed on the computer device. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.

[0123] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is typically used to control the overall operation of a computer device. In this embodiment, the processor is used to run program code stored in memory or process data to implement the processing steps of the cross-domain sentiment classification method based on contrastive learning in Embodiment 1, thereby accurately classifying the sentiment polarity of the user who posted the comment.

[0124] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cross-domain sentiment classification method based on contrastive learning, which enhances the ability of sentiment classification models to extract features from textual information and analyze cross-domain sentiment, characterized in that, The cross-domain sentiment classification method includes the following steps: S1: Construct a cross-domain sentiment training model that includes a source domain feature extractor, a target domain feature extractor, a sentiment classifier, a domain aligner, and a contrastive learner; the data processing method for the cross-domain sentiment training model is as follows: (1) Use the source domain feature extractor to extract features from the source domain data Ds to obtain the source domain features Fs; (2) Perform data augmentation on the target domain data Dt to obtain augmented data Dt′; (3) Use the target domain feature extractor, which shares parameters with the source domain feature extractor, to extract features from the target domain data Dt and the enhanced data Dt′, and obtain the target domain feature Ft and the enhanced feature Ft′, respectively. (4) The output of the source domain feature Fs is divided into two paths. One path is input into the sentiment classifier, which processes the sentiment classifier to obtain the corresponding sentiment polarity classification result; the other path is input into the domain aligner. (5) The target domain feature Ft extracted by the target domain feature extractor is divided into two paths. One path is input to the domain aligner, which calculates the difference between the two domains based on the source domain feature Fs and the target domain feature Ft. The other path is input to the contrast learner. (6) The enhanced feature Ft′ extracted by the target domain feature extractor is input into the contrastive learner, which combines the target domain feature Ft and the enhanced feature Ft′ to extract specific features in the target domain; S2: Obtain a large collection of comment data and preprocess the comment data to form the required comment dataset. The comment dataset includes source domain data Ds and target domain data Dt. The source domain data Ds contains sentiment labels, and the target domain data Dt does not contain labels. S3: Design the overall objective loss function L required for training the cross-domain emotion training model, including the following steps: S31: Based on the classification objective, the cross-entropy loss L... sent Source domain sentiment classification loss as a sentiment classifier; S32: Based on the learning characteristics of discriminative features, the objective function for evaluating the performance of the contrastive learner in extracting specific features of the target domain is set as the contrastive loss L. con ; S33: Perform feature alignment on the source and target domain data to extract domain-invariant features, and then set the loss function of the domain aligner to the alignment difference loss L. align ; S34: Minimize the cross-entropy loss L sent , contrast loss L con Alignment difference loss L align The overall objective loss function L during the training phase of the cross-domain emotion training model is: L=L sent +αL con +βL align Where α and β are the weighting factors for the contrast loss term and the alignment difference loss term, respectively; S4: Train the cross-domain sentiment training model using the Adam optimizer. The training phase includes the following steps: S41: Use the target domain data Dt and part of the source domain data Ds in the comment dataset as the training set; use the remaining source domain data Ds as the validation set; S42: Use the designed overall objective loss function L as the optimization function during the training phase; S43: Set the parameters for the training phase, including: learning rate, number of iterations, batch size, and hyperparameters in the overall objective loss function L; S44: Use the training set to perform batch iterative training on the cross-domain emotion training model; and save the model parameters after each round of training; S45: Using accuracy as the evaluation metric, after each round of training, the trained network model is validated using a validation set, and the corresponding evaluation metric is calculated; the model parameters of the cross-domain sentiment training model with the best evaluation metric are updated and saved. S5: Select the cross-domain sentiment training model with the best evaluation metric after training, take the target domain feature extractor as the feature extraction part, and take the sentiment classifier as the classification part to obtain the required cross-domain sentiment classification network. S6: The cross-domain sentiment classification network is used to extract and analyze the features of the input target text information in sequence, and then output the corresponding cross-domain sentiment classification results.

2. The cross-domain sentiment classification method based on contrastive learning as described in claim 1, characterized in that: In the cross-domain sentiment training model constructed in step S1, both the source domain feature extractor and the target domain feature extractor use the pre-trained language model BERT as the backbone to encode the contextual information of the comment statement; the two feature extractors share parameters.

3. The cross-domain sentiment classification method based on contrastive learning as described in claim 2, characterized in that: In the cross-domain emotion training model constructed in step S1, the emotion classifier consists of a multilayer perceptron (MLP) and a softmax layer; wherein, the multilayer perceptron includes four layers, namely: a fully connected layer, a ReLU activation function layer, a dropout layer, and a fully connected layer; the output representation after passing through the MLP is sent to the softmax layer, and the corresponding loss is calculated therefrom; The domain alignment consists of a multilayer perceptron (MLP) and a contrastive learning layer, with the CMD layer consisting of multiple fully connected layers; the contrastive learner consists of a multilayer perceptron (MLP) and a contrastive learning layer.

4. The cross-domain sentiment classification method based on contrastive learning as described in claim 1, characterized in that: In step S1, back-translation is used to augment the data of each comment text in the target domain data Dt, thereby obtaining positive sample pairs for each comment text.

5. The cross-domain sentiment classification method based on contrastive learning as described in claim 1, characterized in that: In step S2, the preprocessing of the comment corpus is as follows: First, load the comment corpus and a pre-trained language model; the pre-trained language model is either BERT or RoBERTa; then, perform text preprocessing and text data formatting on the comment corpus using the pre-trained language model.

6. The cross-domain sentiment classification method based on contrastive learning as described in claim 5, characterized in that: The steps for text preprocessing and text data formatting in a pre-trained language model are as follows: S001: Use the NLTK tokenizer to segment the comment statement and separate the segmented tokens with spaces; S002: Add two special tokens, [CLS] and [SEP], after the token sequence of the comment statement to construct a general input form: S = {[CLS], w1, w2, ..., w...} n [SEP] Where n represents the total number of tokens in the comment statement, w n This represents the nth token in the comment statement; [CLS] tokens are used for categorization, and [SEP] tokens are used to separate two sentences. S003: Pad the token sequence of each comment statement to make it 256 in length. Statements longer than 256 are truncated, and statements shorter than 256 are padded with zeros. S004: Use the tokenizer of the pre-trained language model to perform a tokenization operation on each token word in the comment statement; obtain the source domain data Ds or target domain data Dt in the required batch data form.

7. The cross-domain sentiment classification method based on contrastive learning as described in claim 1, characterized in that: In step S31, the cross-entropy loss L of the sentiment classifier sent The function is as follows: Where, n s y represents the number of data samples in the original domain. i Let y be the label of the i-th sample in the original domain data. i ′ represents the predicted label of the model on the i-th sample in the original domain data.

8. The cross-domain sentiment classification method based on contrastive learning as described in claim 1, characterized in that: In step S32, the contrastive loss L of the contrast learner con as follows: Where k represents the index of each term in the summation function; z i ,z j sim(z) represents a pair of positive samples; i , z j ) represents z i and z j The cosine similarity; N is the batch size; t is the temperature hyperparameter, which focuses the model update on difficult negative examples and penalizes them accordingly; I is the equality indicator function, which is zero when k = i and one otherwise.

9. The cross-domain sentiment classification method based on contrastive learning as described in claim 1, characterized in that: In step S33, the alignment difference loss L align The function is as follows: Where N represents the number of fully connected layers; and These represent the features of the source domain and the target domain after passing through the i-th fully connected layer, respectively. and These represent the weights of the i-th fully connected layer in the source and target domains, respectively. and These represent the biases of the i-th fully connected layer in the source and target domains, respectively; F represents the ReLU activation function. This is expressed as a K-order central moment calculation function; specifically as follows: Where A and B are two bounded random variables with a range of [a, b]; E(A) and E(B) are the expected values ​​calculated for samples A and B, respectively; C k (A) and C k (B) represents the k-th order central moment, which is the expected value of the distance of the sample from the center to the power of k.

10. The cross-domain sentiment classification method based on contrastive learning as described in claim 1, characterized in that: In step S43, among the parameters set for the training phase, the learning rate is 2.0e-5, and the contrastive loss L in the contrastive learner is... con The temperature parameter is 0.065; the batch size is set to 128; and the number of training iterations is 32.

Images