Cross-domain sample migration aspect level sentiment classification method based on enhanced migration learning

Through reinforced transfer learning and shared encoder, combined with sentence-level sample transfer and reinforced learning, the resource dependence and training efficiency problems in cross-domain level emotional classification are solved, efficient sample selection and model training are achieved, and generalization ability is improved and classification effects.

CN119003771APending Publication Date: 2024-11-22SHANXI UNIV OF FINANCE & ECONOMICS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411046914.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Cross-domain emotional classification tasks face resource dependence, generalization and training efficiency problems. Traditional methods rely on external resources. Word-level domain adaptation is affected by noise. The sample selection process lacks effectiveness, and the training process is highly fragmented and cannot be fully utilized. Downstream task feedback signal.

Method used

Using a cross-domain sample transfer method based on reinforcement transfer learning, low-quality samples are filtered through sentence-level sample transfer, combined with reinforcement learning, the sample selection process is automatically updated, and the sample selection process is jointly carried out with the target model training process, using shared encoder and reinforcement learning. The mechanism improves model generalization ability and training efficiency.

Benefits of technology

It effectively solves the problem of sample selection and distribution differences, improves the effect of cross-domain-level emotional classification, significantly improves the generalization ability and training efficiency of the model, and avoids external resource dependence and noise interference.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003771A_ABST
    Figure CN119003771A_ABST
Patent Text Reader

Abstract

The invention discloses a cross-domain sample migration aspect level sentiment classification method based on enhanced migration learning. In a cross-domain aspect-level sentiment classification task, a cross-domain sample migration method based on enhanced migration learning is provided for the scarcity of heavy label data of a specific-domain aspect-level sentiment classification task. According to the method, firstly, the problems of external resource dependence and excessive word noise are solved on the data level by adopting a sentence-level sample migration method, and meanwhile, low-quality samples in the source field are filtered; secondly, automatically updating a sample selection strategy according to the expressions of a target task and a source field auxiliary task by using enhanced transfer learning in a method level; and finally, a sample selection process and a migration training process of the target model are jointly trained in a training level, so that the training efficiency of the migration model is improved. Experimental results show that the method provided by the invention is superior to a baseline model, and the accuracy of cross-domain aspect level sentiment classification is effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing sentiment analysis and proposes a cross-domain sample transfer aspect-level sentiment classification method based on enhanced transfer learning. Background Art

[0002] The Aspect Level Sentiment Classification (ABSA) task can help users quickly understand the sentiment tendency of key aspects in a text, thereby efficiently filtering and mining useful information. To handle the ABSA task, the earliest methods were mainly based on manually designed rules and features. These methods rely on the knowledge and experience of domain experts to determine the aspect categories and sentiment polarity in the text by defining rules and extracting key features, but this method requires a lot of manual work and cannot adapt to the changes in different domains and tasks. Although deep learning methods have been able to achieve good results on the ABSA task, they are extremely dependent on high-quality labeled training data. However, the ABSA task requires the annotation of the sentiment tendency of each aspect category of the supporting data, which is a very time- and labor-intensive process. In many fields, the supporting data for aspect-level sentiment classification is still very scarce. To solve the problem of data scarcity, the cross-domain ABSA task has emerged. This task obtains potential information from the labeled dataset of the source domain to help train the ABSA task in the target domain where the training data is scarce. This task is receiving widespread attention from academia and industry and has become a promising research direction.

[0003] Currently, researchers mostly use transfer learning to solve cross-domain ABSA problems. For cross-domain ABSA tasks, early studies mostly used the common syntactic relationship between aspect words and opinion words to make up for the gap in the field, but this is highly dependent on external language resources. In order to get rid of this dependence, researchers adopted the word-level domain adaptation method, but due to the large interference of noise words, the effect is not outstanding. In order to avoid this problem, transfer learning methods based on sample selection have attracted much attention. How to deal with the domain differences between different fields and thus select source domain samples suitable for transfer is extremely important.

[0004] Most sample selection methods consider weighting the training data under a certain metric, and then select source domain samples based on a preset metric threshold to assist in the training of the target model. However, the generalization performance of such sample selection methods based on preset thresholds is generally poor, and they cannot be well applied to other tasks or other fields. In addition, the sample selection process of such methods is separate from the training process of the transfer model, which requires additional steps to train after selecting the source domain samples in order to obtain a model that is transferred to the target domain, and the lack of feedback signals from downstream tasks actually creates difficulties in the process of selecting suitable source domain samples.

[0005] The cross-domain aspect-level sentiment classification task currently faces three key problems: (1) Resource dependency problem. Traditional cross-domain ABSA methods are relatively dependent on external resources such as syntactic relations. Methods based on word-level domain adaptation cannot achieve the expected results in dealing with cross-domain ABSA tasks due to the influence of noise. (2) Generalization problem. Previous sample selection methods based on preset thresholds have poor generalization performance when dealing with cross-domain ABSA tasks. The sample selection process has a large human intervention component, and the selection results lack good interpretability. (3) Training efficiency problem. Previous cross-domain ABSA methods based on sample selection have defects in model training. There is a disconnect between the sample selection process and the transfer learning process of model training, and the feedback signals of downstream tasks are not fully utilized.

[0006] Reinforcement learning is an effective framework for solving continuous decision-making tasks, in which an agent continuously interacts with the environment to improve its performance through trial and error. In recent years, reinforcement learning has made remarkable progress under the rapid development of deep neural networks. With the broad prospects of reinforcement learning in many fields such as robotics and game theory, researchers have begun to use reinforcement learning to promote transfer learning by transferring knowledge from external fields.

[0007] Aiming at the cross-domain ABSA task, the present invention innovates from three levels: data, method, and training, and proposes a cross-domain sample transfer aspect-level sentiment classification method (RTSSM) based on reinforcement transfer learning. Specifically, the method first adopts sentence-level sample migration to solve the problems of external resource dependence and excessive word noise at the data level, and at the same time filters out low-quality samples in the source domain. Secondly, at the method level, reinforcement transfer learning is used to automatically update the sample selection strategy according to the target task performance. Finally, at the training level, the sample selection process is combined with the migration training process of the target model to improve the training efficiency of the migration model. The cross-domain sample transfer aspect-level sentiment classification method based on reinforcement transfer learning proposed in the present invention effectively solves the problems of sample selection and distribution differences in cross-domain aspect-level sentiment classification tasks, and can effectively improve the effect of cross-domain aspect-level sentiment classification tasks. Summary of the invention

[0008] The present invention aims to solve the cross-domain aspect-level sentiment classification task by combining sample transfer method and reinforcement learning mechanism.

[0009] To achieve the above object, the present invention proposes a cross-domain sample transfer aspect-level sentiment classification method based on enhanced transfer learning, comprising the following steps:

[0010] S1, the overall framework of the cross-domain sample transfer aspect-level sentiment classification method based on reinforcement transfer learning; the framework mainly includes three parts: data processing and shared encoder module, aspect-level sentiment classification module and sample selection module based on reinforcement learning;

[0011] S2, data processing and shared encoder module; in the data processing part, 150 samples in the target domain are manually annotated as a guidance set, and the source domain data is divided into blocks for subsequent iterative training; in the shared encoder part, a Transformer-based shared encoder is used to generate embedded representations for the samples, so that the encoder learns a unified representation in the source and target domains, thereby understanding and processing data in different domains;

[0012] S3, aspect-level sentiment classification module; the module designs two sentiment classification tasks: the first is the document-level sentiment classification task on the source domain, and the second is the aspect-level sentiment classification task on the target domain; the two tasks share the underlying features through the shared encoder module, and influence and enhance each other's performance;

[0013] S4, a sample selection module based on reinforcement learning; a sample selection probability generator is designed based on reinforcement learning to guide sample selection, and feedback is provided to the sample selection probability generator by comprehensively considering the downstream task performance and domain differences. A sample selection strategy is used to select samples from the source domain for transfer training of the target model.

[0014] According to the embodiment of the present invention, a cross-domain sample transfer aspect-level sentiment classification method based on reinforcement transfer learning is proposed; the present invention first uses a shared encoder as a text encoding module, and designs a document-level sentiment classifier on the source domain and an aspect-level sentiment classifier on the target domain respectively; secondly, a sample selection strategy is generated through a sample selection probability generator, and feedback is provided to the sample selection probability generator by comprehensively considering the target model performance and domain differences; finally, the screened source domain samples and the labeled target domain samples are used for joint training, and the training of the target model is completed simultaneously during the sample selection strategy training process; experimental results show that the method proposed by the present invention is superior to the baseline model and effectively improves the results of cross-domain aspect-level sentiment classification.

[0015] According to one embodiment of the present invention, step S2 further includes:

[0016] S21, manual data annotation: randomly select 150 data from the training data of the target domain and manually annotate them as the target domain guidance set data, and divide the source domain data set into source domain data blocks according to the size of the target domain guidance set; in each iteration, the model processes a source domain data block and the data of the target domain guidance set, avoiding the problem of the model being biased towards the source domain data due to imbalance of the two domains caused by too much source domain data, alleviating GPU pressure and improving training efficiency;

[0017] S22, the source domain auxiliary task is the document-level sentiment classification task, and the initial sentiment labels of the texts in the source domain dataset are sentiment labels related to the aspect categories; the source domain dataset is subjected to label statistical re-labeling, and the majority labels of the aspect category sentiment labels of each text are used as the document-level sentiment labels of the text, and texts with equal positive and negative characteristics are labeled as conflicting, and texts without sentiment tendency are labeled as neutral; the source domain document-level dataset obtained after re-labeling is is the text information of the source domain sample, where is the document-level sentiment label after re-annotation, N S is the number of samples in the source domain dataset;

[0018] S23, in order to facilitate the model to encode the input sequence and share feature extraction, the Transformer-based shared encoder module uses hard parameter sharing to share model parameters between different input sequences; the module consists of encoder layers, each encoder layer contains a multi-head self-attention mechanism and a feedforward neural network module, and the output of the module is recorded as E:

[0019] E=Encoder(Encoder(Encoder(X)))

[0020] Among them, Encoder() represents an encoder layer and X is the input sample.

[0021] According to one embodiment of the present invention, step S3 further includes:

[0022] S31, the document-level sentiment classification model on the source domain receives the sentence-level feature representation E obtained by the encoder output of the source domain sample S As the model input, the perceptron model is used to transform the text representation features to obtain the sentiment classification results; and the cross entropy loss function is used to calculate the source domain model loss in each sample selection round based on the prediction results:

[0023]

[0024] Among them, Q is the number of source domain data blocks, is the source domain model loss of data block b;

[0025] S32, the aspect-level sentiment classification model in the target domain receives the guidance set text and obtains the word-level feature vector matrix E output by the encoder T ={e1,e2,…,e n} as input, use the long short-term memory network to transform the text representation and use the four-classification perceptron model to get the aspect-level sentiment label prediction results of each word, and get the aspect-level sentiment label sequence of the text; the model uses the cross entropy loss function to calculate the target domain model loss in each sample selection round according to the prediction results:

[0026]

[0027] Among them, Q is the number of source domain data blocks, is the target domain model loss of data block b.

[0028] According to one embodiment of the present invention, step S4 further includes:

[0029] S41, under the initial sample selection strategy, in each training iteration, a source domain sample data block B and a target domain sample set T are sequentially input into the encoder until all source domain data blocks are processed; the encoder outputs the sentence feature representation set B of B and T respectively. E 、T E , perform average pooling operations on the sets respectively, and obtain the average vector H of the two sets B , H T ; After processing all data, calculate the total difference measure DIS:

[0030]

[0031] Among them, Q is the number of source domain data blocks, dis i represents the set difference of the i-th iteration;

[0032] S42, based on the source domain model loss , target domain model loss And the total set difference DIS calculates the loss function of the reinforcement learning module:

[0033]

[0034] Among them, α and β are weight coefficients, are the sum of the losses of the source domain model and the target domain model in Q iterations under this round of sample selection strategy;

[0035] S43, the sample selection strategy is given by the sample selection probability generator, which is implemented by the perceptron model. The input and output are both n-dimensional vectors; the input is an n-dimensional vector P = {p1, p2, ..., p i ,…,p n}, p i represents the probability of the i-th sample of the source domain data block being selected, and n is the number of samples in the data block; the sample selection probability generator is based on LReward Perform back propagation to update parameters and generate a new sample selection probability vector P new , and according to P new The selection probability is used to sample and implement the sample selection strategy.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] (1) This paper proposes a novel sample transfer cross-domain aspect-level sentiment classification method based on reinforcement transfer learning. This method innovatively applies the idea of ​​reinforcement transfer learning to the cross-domain aspect-level sentiment classification task. This method can not only learn shared language representations from the source domain and the target domain, alleviate the problem of scarce labeled data in the target domain, and significantly improve the generalization ability of the model. In addition, this framework uses the reinforcement learning mechanism to regulate the sample selection strategy in the sample transfer process, making the sample selection process intelligent and further improving the training efficiency of the model.

[0038] (2) In order to avoid the problem of poor effect of updating the encoder module parameters using only the loss of the target model due to sparse target domain samples during the training of the shared encoder module, and to further improve the model performance in the target domain, the present invention designs two downstream tasks: one is the document-level sentiment classification task in the source domain, and the other is the aspect-level sentiment classification task in the target domain; they share the underlying sentiment features through the shared encoder module, and influence and enhance each other's performance; through this sharing, the model uses the domain-independent knowledge learned in the document-level sentiment classification task in the source domain to improve the performance of the aspect-level sentiment classification task in the target domain.

[0039] (3) In order to get rid of the model's dependence on external grammatical resources and avoid the interference of noise words, the present invention performs sample migration at the sentence level; the method comprehensively considers the performance and domain differences of the target domain model and the source domain model to provide feedback for the sample selection probability generator, and processes the source domain data in batches; the loss of each batch of data is used to optimize the classifier and participate in the composition of the total loss of reinforcement learning. The sample selection process and the migration training process of the target model are jointly trained to improve the training efficiency of the migration model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flowchart of the aspect-level sentiment classification method of cross-domain sample transfer based on reinforcement transfer learning according to an embodiment of the present invention.

[0041] Figure 2 This is the overall framework diagram of the cross-domain sample transfer aspect-level sentiment classification method based on reinforcement transfer learning.

[0042] Figure 3It is a framework diagram of the source domain document-level sentiment classification model.

[0043] Figure 4 It is a framework diagram of the target domain aspect-level sentiment classification model. DETAILED DESCRIPTION

[0044] The present invention will be further described below in conjunction with the accompanying drawings.

[0045] like Figure 1 As shown, the framework of the present invention is mainly divided into the following four steps, which are connected layer by layer and finally integrated. The learning process mainly includes the following steps:

[0046] S1, the overall framework of the cross-domain sample transfer aspect-level sentiment classification method based on reinforcement transfer learning; the framework mainly includes three parts: data processing and shared encoder module, aspect-level sentiment classification module and sample selection module based on reinforcement learning;

[0047] S2, data processing and shared encoder module; in the data processing part, 150 samples in the target domain are manually annotated as a guidance set, and the source domain data is divided into blocks for subsequent iterative training; in the shared encoder part, a Transformer-based shared encoder is used to generate embedded representations for the samples, so that the encoder learns a unified representation in the source and target domains, thereby understanding and processing data in different domains;

[0048] The target data labeling, source domain data block data processing operation and transformer-based shared encoder in step S2 further include:

[0049] S21, manual data annotation: randomly select 150 data from the training data of the target domain and manually annotate them as the target domain guidance set data, and divide the source domain data set into source domain data blocks according to the size of the target domain guidance set; in each iteration, the model processes a source domain data block and the data of the target domain guidance set, avoiding the problem of the model being biased towards the source domain data due to imbalance of the two domains caused by too much source domain data, alleviating GPU pressure and improving training efficiency;

[0050] S22, the source domain auxiliary task is the document-level sentiment classification task, and the initial sentiment labels of the texts in the source domain dataset are sentiment labels related to the aspect categories; the source domain dataset is subjected to label statistical re-labeling, and the majority labels of the aspect category sentiment labels of each text are used as the document-level sentiment labels of the text, and texts with equal positive and negative characteristics are labeled as conflicting, and texts without sentiment tendency are labeled as neutral; the source domain document-level dataset obtained after re-labeling is is the text information of the source domain sample, where is the document-level sentiment label after re-annotation, N S is the number of samples in the source domain dataset;

[0051] S23, in order to facilitate the model to encode the input sequence and share feature extraction, the Transformer-based shared encoder module uses hard parameter sharing to share model parameters between different input sequences; the module consists of encoder layers, each of which contains a multi-head self-attention mechanism and a feedforward neural network module; the encoding process processed by each encoder layer is as follows:

[0052] (1) Multi-Head Self-Attention:

[0053] Input set D S A sample is X, and the input sequence is represented as X = {x1, x2, ..., x i ,…,x n}, where x i is the embedding vector of the i-th word in the sequence;

[0054] The self-attention mechanism uses three linear transformations Query(X) = X × W Q 、Key(X)=X×W K 、Value(X)=X×W v To compute the query, key and value representation; where W Q , W K , W V is a learnable weight matrix, and the results of the three linear transformations are recorded as Q, K, and V respectively; and through To calculate the attention score, where d k is the dimension of the query or key;

[0055] In the multi-head attention layer, 8 sets of query, key and value weight matrices are used to obtain 8 sets of attention scores, and the 8 attention scores are concatenated to obtain the final output M-Attention(X);

[0056] (2) Feedforward Neural Network:

[0057] The model is connected to a feedforward neural network after the multi-head self-attention layer to perform nonlinear transformation on the representation of each position; the output of the multi-head attention layer M-Attention(X) undergoes two linear transformations and the activation function ReLU in the feedforward neural network layer to output the encoding result of the encoder of this layer:

[0058] Encoder(X)=FNN(M-Attention(X))

[0059] =ReLU(M-Attention(X)×W1+b1)×W2+b2

[0060] Among them, W1, W2 are learnable weight matrices, b1, b2 are bias vectors;

[0061] The entire encoder module is composed of 3 encoder layers stacked together, and the output of the module is denoted by E:

[0062] E=Encoder(Encoder(Encoder(X)))

[0063] Among them, Encoder() represents an encoder layer, and X is the input sample;

[0064] In this module, for each encoder layer, the weight matrix W Q , W K , W V , W1, W2 are of fixed size and are shared in all input sequences in the source and target domains; the model can process sequence data of indefinite length, and the design of sharing the underlying encoder parameters enables the encoder to learn a unified representation in the source and target domains, and can understand and process data in different domains, reducing the model size and improving processing efficiency;

[0065] S3, aspect-level sentiment classification module; the module designs two sentiment classification tasks: one is the document-level sentiment classification task in the source domain, and the other is the aspect-level sentiment classification task in the target domain; the two tasks share the underlying features through the shared encoder module, and influence and enhance each other's performance;

[0066] The document-level sentiment classification task on the source domain and the aspect-level sentiment classification task on the target domain in step S3 further include:

[0067] S31, the framework diagram of the document-level sentiment classification model in the source domain is as follows Figure 3 As shown, the model receives the source domain sample and obtains the sentence-level feature representation E output by the encoder S As the model input, the multi-layer perceptron model is used to transform the features of the text representation to obtain the sentiment classification result; the multi-layer perceptron model contains two hidden layers, the number of neurons in each hidden layer is h1 and h2, and the output layer contains 4 neurons, indicating the number of sentiment categories;

[0068] The model uses the cross entropy loss function to measure the difference between the predicted probability and the true label of the sample. The source domain model loss for each data block is:

[0069]

[0070] in, is the probability that the i-th sample belongs to category c predicted by the model; is the probability (0 or 1) that the i-th sample in the true label belongs to category c; N B is the number of samples in the source domain data block;

[0071] In each sample selection round, the loss of the source domain document-level sentiment classification model is:

[0072]

[0073] Among them, Q is the number of source domain data blocks, is the source domain model loss of data block b;

[0074] S32, the framework of the aspect-level sentiment classification model in the target domain is shown in Figure 4 As shown, the model receives the guidance set text and obtains the word-level feature vector matrix E output by the encoder T ={e1,e2,…,e n} as input, use the long short-term memory network to transform the text representation and use the four-classification multi-layer perceptron model to get the aspect-level sentiment label prediction result of each word, so as to get the aspect-level sentiment label sequence of the text;

[0075] Input data E T In a long short-term memory network unit, the forget gate f t =σ(W f ·[h t-1 ,e t ]+b f ), input gate i t =σ(W i ·[h t-1 ,e t ]+b i ), Processing, then update the cell status And by the output gate o t =σ(W o ·[h t-1 ,e t ]+b o ),h t =o t *tanh(C t ) Output hidden state;

[0076] Among them, e t represents the word vector of the tth time step in the input sequence, σ represents the Sigmoid activation function, tanh represents the hyperbolic tangent activation function, and W and b are network parameters;

[0077] After passing through multiple long short-term memory network units, the hidden state output sequence of the last time step is used as the word feature vector matrix of the text, represented by R T ={r1,r2,…,r n}; i (i=1,2,…,n) are used as the input of the four-classification multilayer perceptron model in turn; the multilayer perceptron model contains two hidden layers, the number of neurons in each hidden layer is h3 and h4, and the output layer contains 4 neurons, representing the labels "O", "T-POS", "T-NEG", and "T-NEU" respectively;

[0078] The model uses the cross entropy loss function to measure the difference between the predicted result and the true sentiment label. The model loss of each sentence sample and the model loss of the target domain guidance set are:

[0079]

[0080] in, is the probability that the i-th word belongs to aspect sentiment label j predicted by the model; is the probability (0 or 1) that the i-th word in the true label belongs to aspect sentiment label j, len is the length of the sentence, N G is the number of samples in the guidance set;

[0081] In each sample selection round, the target domain aspect-level sentiment classification model loss is:

[0082]

[0083] Among them, Q is the number of source domain data blocks, is the target domain model loss of data block b;

[0084] S4, sample selection module based on reinforcement learning; based on reinforcement learning, a sample selection probability generator is designed to guide sample selection, and feedback is provided to the sample selection probability generator by comprehensively considering downstream task performance and domain differences. A new sample selection strategy is used to select high-value samples from the source domain for transfer training of the target model;

[0085] The sample selection module based on reinforcement learning in step S4 further comprises:

[0086] S41, under the initial selection strategy, the source domain sample data block input to the encoder in each iteration is The target domain sample set is The sentence feature representation set output by the encoder is Perform average pooling operations on the sets respectively to obtain the average vector H of the two sets B , HT , then the calculation formula of the difference measure between the two sets is as follows:

[0087]

[0088] in Represents H B , H T The value of the i-th dimension of , V represents the number of dimensions of the vector; each round of sample selection strategy processes a total of Q data blocks of data, and the total set difference after Q iterations is as follows:

[0089]

[0090] Among them, Q is the number of source domain data blocks, dis i represents the set difference of the i-th iteration;

[0091] S42, based on the source domain model loss Target Domain Model Loss And the total set difference DIS calculates the loss function of the reinforcement learning module:

[0092]

[0093] Among them, α and β are weight coefficients, are the sum of the losses of the source domain model and the target domain model in Q iterations under this round of sample selection strategy;

[0094] S43, the sample selection strategy is given by the sample selection probability generator, which is implemented by a multi-layer perceptron model. Both the input and output are n-dimensional vectors; the input is an n-dimensional vector P = {p1, p2, ..., p i ,…,p n}, p i represents the probability of the i-th sample of the source domain data block being selected, and n is the number of samples in the data block; the multilayer perceptron model contains 2 hidden layers, and the output H of each hidden layer l (l) It can be calculated as follows:

[0095] H (l) =ReLU(W (l) ·H (l-1) +b (l) )

[0096] Among them, W (l) is the weight matrix of hidden layer l, H (l-1) is the output of the previous layer (input layer or previous hidden layer), b (l) is the bias vector of hidden layer l;

[0097] The output H of the last hidden layer (m)As the input of the output layer, the final output P is obtained through the Sigmoid activation function. new ,in:

[0098] P new =Sigmoid(W (out) ·H (m) +b (out) )

[0099] Where P new is also an n-dimensional vector, W (out) is the weight matrix of the output layer, b (out) is the bias vector of the output layer; Sigmoid is the activation function of the output layer, which normalizes the multilayer perceptron output to probability to obtain the sample selection probability vector P new ; The sample selection probability generator is based on L Reward Perform back propagation to update parameters and generate a new sample selection probability vector P new , and according to P new The selection probability is used to perform sampling and implement the sample selection strategy;

[0100] In order to verify the effectiveness of the proposed method in cross-domain aspect-level sentiment classification, different baseline models were selected for comparison; all baseline models were divided into four categories: (1) rule-based methods, including TCRF and RAP; (2) deep learning-based methods, including Hier-Joint, RNSCN, Hier-Joint+, and RNSCN+; (3) adversarial learning-based methods, including SAL and AHF; (4) reinforcement learning-based methods, including the method proposed in this invention, RTSSM. The detailed description of the above baseline models is as follows:

[0101] TCRF: a traditional sequence model that combines CRF with hand-designed rules such as POS tags, short dependency paths, and word distances for domain adaptation;

[0102] RAP: A relation-adaptive bootstrapping algorithm that uses a multi-domain common vocabulary of aspect terms and syntactic rule relations to extend domain-specific aspect vocabularies;

[0103] Hier-Joint: Uses a rule-based unsupervised approach to create auxiliary labels and a long short-term memory neural network model to learn hidden representations applicable to different domains;

[0104] RNSCN: A recurrent neural network for cross-domain aspect extraction that effectively reduces domain shift at the word level via syntactic relations and generates an auxiliary task to predict the relationship between any two adjacent words in a syntactic dependency tree;

[0105] Hier-Joint+ / RNSCN+: Extended Hier-Joint / RNSCN with an additional long short-term memory layer on top of the source model to predict vertical and horizontal boundary labels;

[0106] SAL: A cross-domain aspect-level sentiment classification method using dual memory interaction and selective adversarial learning; A new selective adversarial learning (SAL) method that dynamically learns the weight of each word so that important words can have high weights, thereby achieving fine-grained (word-level) adaptation;

[0107] AHF: An adaptive hybrid framework that utilizes pseudo labels generated on target domain data to train task classifiers and aligns feature distributions through adversarial training, combining semi-supervised learning and adversarial training for cross-domain aspect-level sentiment analysis;

[0108] RTSSM(ours): A sample selection method based on reinforcement transfer learning. It uses reinforcement learning to guide the sample selection process of transfer learning and completes the transfer training of the target model during the training of the sample selection strategy.

[0109] The comparison results of the F1 scores of the method proposed in the present invention and the baseline model are shown in Table 1. It can be found from the table that the average F1 value is gradually improving in the order of rule-based method, deep learning-based method, adversarial learning-based method, and reinforcement learning-based method.

[0110] Table 1 Comparison results of F1 value (%) between RTSSM and baseline model

[0111] Model D→R D→S L→R L→S R→D R→L R→S S→D S→L S→R Avg TCRF 17.05 13.49 16.06 12.34 19.84 14.59 15.20 13.43 9.56 14.84 14.64 RAP 28.37 16.80 31.05 13.72 17.50 15.69 13.17 15.74 12.38 25.41 18.98 Hier-Joint 30.03 18.74 31.90 15.33 22.91 19.17 15.20 20.04 21.80 32.81 22.79 <![CDATA[Hier-Joint + ]]> 32.87 19.04 33.54 13.90 24.53 20.72 15.56 23.24 22.65 31.10 23.72 RNSCN 31.41 18.93 31.85 16.73 32.43 25.54 23.31 19.98 19.15 30.56 24.99 <![CDATA[RNSCN + ]]> 34.60 20.03 35.65 16.59 33.26 26.63 20.04 22.00 18.87 33.21 26.09 SAL 41.64 30.34 42.60 28.00 36.36 32.36 30.14 35.97 26.46 42.18 34.61 AHF 44.57 34.96 43.49 33.05 37.33 34.89 33.23 39.61 29.01 46.55 37.67 RTSSM 42.68 41.78 41.07 45.63 48.96 45.46 43.28 47.01 45.80 43.38 44.75

[0112] (1) Compared with the traditional rule-based TCRF and RAP methods, the latter methods based on deep learning, adversarial learning, and reinforcement learning have achieved superior results in all subtasks; this shows that it is not easy for rule-based methods to find rules that are universal to all domains in multiple cross-domain migration tasks; the other three methods can capture the intrinsic distribution of data to a certain extent through neural networks, which makes them have high generalization ability when facing unseen data, thus achieving superior results than rule-based methods;

[0113] (2) Among the deep learning-based methods, Hier-Joint, RNSCN, and their respective enhanced variant models all perform worse than the adversarial learning-based methods; the adversarial learning-based SAL method outperforms the two enhanced variant models Hier-Joint and RNSCN. + and RNSCN +The average F1 value is 10.89% and 8.52% higher respectively; this is because Hier-Joint + , RNSCN + The models all learn cross-domain feature representations from the perspective of finding common grammatical dependencies between different domains. However, noise is inevitable in the cross-domain grammatical dependency parse trees, which will affect their performance. The SAL method based on adversarial learning uses selective adversarial training to dynamically allocate word weights to capture domain-shared feature representations, getting rid of the limitations of grammatical dependencies and the interference of noise in grammatical dependency trees.

[0114] (3) Among the methods based on adversarial learning, the performance of the AHF method is better than that of the SAL method, with an average F1 value of 3.06% higher. This is because the SAL method only trains on the source domain data to assign word weights, and does not make full use of the target domain information. On the other hand, the AHF method uses the target domain data to train the domain discriminator and the target domain classifier simultaneously based on the SAL method. Compared with the SAL method, the domain shared information extracted can contain the target domain information.

[0115] (4) As can be seen from the table, the proposed RTSSM method is superior to the SAL and AHF methods based on adversarial learning. This is because although the SAL and AHF methods perform domain adaptation at the word level through weight allocation and get rid of grammatical relationship dependence and noise interference, the two methods still use all the source domain data to support the transfer task. According to the sample selection theory, not all data in these source domain data play a positive role in the transfer task. The data in the source domain data will hinder the training of the transfer task due to the existence of domain differences. Compared with the strategy of SAL and AHF methods that screen at the word level and then perform domain adaptation, the proposed RTSSM method adopts an effective sample level selection strategy. Through reinforcement learning, the RTSSM method directly updates the sample selection strategy based on the target domain task performance and the source domain auxiliary task performance, and gradually filters out low-quality samples in the source domain data. Finally, the RTSSM method improves the average F1 value by 10.14% and 7.08% compared with the SAL method and the AHF method, respectively, which verifies the effectiveness of the proposed RTSSM method.

[0116] In summary, the present invention proposes a cross-domain sample transfer aspect-level sentiment classification (RTSSM) method based on reinforced transfer learning; the RTSSM method updates the sentence-level sample selection strategy according to the performance feedback of the target task by means of reinforced transfer learning, avoiding dependence on external resources and reducing the interference caused by word noise; the designed source domain auxiliary task makes up for the disadvantage of small data in the target task, effectively guides the update of the sample selection strategy, and filters low-quality samples in the source domain; the method jointly trains the sample selection mechanism with the training process of target model migration, which significantly improves the training efficiency of the migration model; the migration task experiment on the ABSA dataset proves that the proposed RTSSM method is superior to the current advanced methods in 7 migration subtasks and average performance; through the proposed method, a balance is achieved between the quantity and quality of source domain data, so that the training effect of the migration task is improved while also improving the efficiency of model training;

[0117] The explanation in this article combined with the drawings and specific embodiments of the specification is only used to help understand the method and core idea of ​​the present invention; the method described in the present invention is not limited to the embodiments described in the specific implementation manner, and other implementation methods derived by those skilled in the art based on the method and idea of ​​the present invention also belong to the technical innovation scope of the present invention; the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A cross-domain sample transfer aspect-level sentiment classification method based on enhanced transfer learning, characterized in that: The following steps are involved: S1, the overall framework of the cross-domain sample transfer aspect-level sentiment classification method based on reinforcement transfer learning; the framework mainly includes three parts: data processing and shared encoder module, aspect-level sentiment classification module and sample selection module based on reinforcement learning; S2, data processing and shared encoder module; in the data processing part, 150 samples in the target domain are manually annotated as a guidance set, and the source domain data is divided into blocks for subsequent iterative training; in the shared encoder part, a Transformer-based shared encoder is used to generate embedded representations for the samples, so that the encoder learns a unified representation in the source and target domains, thereby understanding and processing data in different domains; S3, aspect-level sentiment classification module; The module designs two sentiment classification tasks: the first is the document-level sentiment classification task on the source domain, and the second is the aspect-level sentiment classification task on the target domain; The two tasks share the underlying features through the shared encoder module and influence and enhance each other's performance; S4, sample selection module based on reinforcement learning; A sample selection probability generator is designed based on reinforcement learning to guide sample selection. The downstream task performance and domain differences are comprehensively considered to provide feedback for the sample selection probability generator. A sample selection strategy is used to select samples from the source domain for transfer training of the target model.

2. According to claim 1, a cross-domain sample transfer aspect-level sentiment classification method based on enhanced transfer learning is characterized in that: The data processing and shared encoder module in step S2 further comprises: S21, manual data annotation: randomly select 150 data from the training data of the target domain and manually annotate them as the target domain guidance set data, and divide the source domain data set into source domain data blocks according to the size of the target domain guidance set; During each iteration, the model processes a source domain data block and a target domain guidance set data, avoiding the problem of the model being biased towards the source domain data due to imbalance of the two domains caused by too much source domain data, alleviating GPU pressure and improving training efficiency; S22, the source domain auxiliary task is a document-level sentiment classification task, and the initial sentiment labels of the texts in the source domain dataset are sentiment labels related to aspect categories; Perform label statistical re-labeling on the source domain dataset, take the majority label of the aspect category sentiment label of each text as the document-level sentiment label of the text, label the text with equal positive and negative characteristics as conflicting, and label the text with no sentiment tendency as neutral; The document-level dataset of the source domain obtained after re-annotation is is the text information of the source domain sample, where is the document-level sentiment label after re-annotation, N S is the number of samples in the source domain dataset; S23, in order to facilitate the model to encode the input sequence and share feature extraction, the Transformer-based shared encoder module uses hard parameter sharing to share model parameters between different input sequences; the module consists of encoder layers, each encoder layer contains a multi-head self-attention mechanism and a feedforward neural network module, and the output of the module is recorded as E: E=Encoder(Encoder(Encoder(X))) Among them, Encoder() represents an encoder layer and X is the input sample.

3. According to claim 1, a cross-domain sample transfer aspect-level sentiment classification method based on enhanced transfer learning is characterized in that: The aspect-level sentiment classification module in step S3 further comprises: S31, the document-level sentiment classification model on the source domain receives the sentence-level feature representation E obtained by the encoder output of the source domain sample S As the model input, the perceptron model is used to transform the text representation features to obtain the sentiment classification results; and the cross entropy loss function is used to calculate the source domain model loss in each sample selection round based on the prediction results: Among them, Q is the number of source domain data blocks, is the source domain model loss of data block b; S32, the aspect-level sentiment classification model in the target domain receives the guidance set text and obtains the word-level feature vector matrix E output by the encoder T ={e1,e2,…,e n } as input, use the long short-term memory network to transform the text representation and use the four-classification perceptron model to get the aspect-level sentiment label prediction results of each word, and get the aspect-level sentiment label sequence of the text; the model uses the cross entropy loss function to calculate the target domain model loss in each sample selection round according to the prediction results: Among them, Q is the number of source domain data blocks, is the target domain model loss of data block b.

4. According to claim 1, a cross-domain sample transfer aspect-level sentiment classification method based on enhanced transfer learning is characterized in that: The sample selection module based on reinforcement learning in step S4 further comprises: S41, under the initial sample selection strategy, in each training iteration, one source domain sample data block B and one target domain sample set T are sequentially input into the encoder until all source domain data blocks are processed; The encoder outputs the sentence feature representation set B of B and T respectively E 、T E , perform average pooling operations on the sets respectively, and obtain the average vector H of the two sets B , H T ; After processing all data, calculate the total difference measure DIS: Among them, Q is the number of source domain data blocks, dis i represents the set difference of the i-th iteration; S42, based on the source domain model loss Target Domain Model Loss And the total set difference DIS calculates the loss function of the reinforcement learning module: Among them, α and β are weight coefficients, are the sum of the losses of the source domain model and the target domain model in Q iterations under this round of sample selection strategy; S43, the sample selection strategy is given by the sample selection probability generator, which is implemented by the perceptron model. The input and output are both n-dimensional vectors; the input is an n-dimensional vector P = {p1, p2, ..., p i ,…,p n }, p i represents the probability of the i-th sample of the source domain data block being selected, and n is the number of samples in the data block; the sample selection probability generator is based on L Reward Perform back propagation to update parameters and generate a new sample selection probability vector P new , and according to P new The selection probability is used to sample and implement the sample selection strategy.