A cross-language attribute-level sentiment classification method based on translation matching
Through the combination of adversarial domain adaptation training and multi-head self-attention layer, the problem of resource scarcity and fine-grained interaction in cross-language attribute-level sentiment analysis is solved, and the performance improvement of cross-language emotion classification is achieved, especially in the case of resource scarcity. The effect of emotional classification has been significantly improved.
Patent Information
- Application Number
- CN202111529633.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-14
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-12-14
AI Technical Summary
The research on cross-language attribute-level sentiment analysis is still in its infancy, especially in small languages with scarce resources. The dictionary migration method has limitations, the feasibility of the machine translation method has not been fully explored, and the fine-grained emotional classification task is difficult to effectively implement. .
A cross-language attribute-level emotion classification method based on translation matching is proposed, using a model combining adversarial domain adaptation training and attribute emotion classification training, using target encoder, domain classifier, gradient inversion layer, multi-head self-attention layer and Adaptive fusion layer reduces the deviation between target translation and real language through adversarial domain adaptation training, and obtains attribute-level sentence representations through multi-head self-attention layer modeling fine-grained interactions.
It improves the attribute-level emotion classification performance of the target language with scarce corpus resources, reduces the distribution deviation caused by machine translation, can better utilize the rich source language corpus resources, and improves the accuracy and efficiency of emotion classification.
Smart Images

Figure CN114238636B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a natural language processing method, and in particular to a cross-language attribute-level sentiment classification method based on translation matching. Background Art
[0002] There are two main approaches to the coarse-grained cross-language text classification task: one is through bilingual dictionaries and machine translation. For example, Bel et al. initially used bilingual dictionaries to translate the target language into the source language in "Cross-lingual text categorization", and then used a classifier based on the source language to classify the translated text. The second is cross-language representation learning, that is, to achieve cross-language feature alignment at a low cost by transferring cross-language models through a shared feature space. In cross-language word embeddings, words with similar meanings in different languages have similar vector representations. For example, Klementiev et al. used a large amount of parallel corpus to train cross-language word embeddings in "Inducing crosslingual distributed representations of words", and achieved good results on text classification datasets with English as the source language and German as the target language.
[0003] Sentiment classification of text can also be regarded as text classification, and early cross-language sentiment classification research usually adopts the dictionary transfer method, that is, first transfer the source language's sentiment dictionary to the target language, and then build a classifier based on sentiment words in the target language to achieve the purpose of classification. However, when the target language is a small language with scarce resources, bilingual sentiment resources will be difficult to obtain. The method based on dictionary transfer has certain limitations, while sentiment analysis based on machine translation methods is feasible. Banea et al. used a machine translation system to compare the performance of forward translation and reverse translation in subjective sentiment analysis tasks in "Multilingual subjectivity analysis using machine translation", and preliminarily explored the feasibility of sentiment analysis based on machine translation methods.
[0004] Aspect-level sentiment analysis is a fine-grained sentiment classification that relies not on the whole sentence but on a group of words or phrases related to a specific attribute. Currently, the research on cross-language attribute-level sentiment analysis is still in its infancy. Lambert et al. proposed a constrained SMT-based method in "Aspect-level cross-lingual sentiment classification with constrained SMT", which translates sentiment units while considering the context, where the sentiment unit contains sentiment holders and sentiment targets. Experimental results show that the cross-language sentiment classifier established based on this method can achieve results comparable to those of a single-language classifier. Barnes et al. explored the performance of machine translation-based methods and cross-language word embedding representation methods on cross-language attribute-level sentiment classification tasks in "Exploring distributional representations and machine translation for aspect-based cross-lingual sentiment classification". This method uses SVM as a sentiment classifier for performance comparison and analysis. Summary of the invention
[0005] In view of the shortcomings of the prior art, the present invention proposes a cross-language attribute-level sentiment classification method based on translation matching. Through a cross-language attribute-level sentiment classification model based on translation matching with a domain classifier, a target encoder, a gradient reversal layer, a multi-head self-attention layer, and adaptive fusion, the source language with rich corpus resources is used to improve the performance of attribute-level sentiment classification in the target language with scarce corpus resources.
[0006] In order to achieve the above purpose, the technical scheme of the present invention is as follows: a cross-language attribute-level sentiment classification method based on translation matching can be divided into two parts: adversarial domain adaptation training and attribute sentiment classification training, including a target encoder, a domain classifier, a gradient reversal layer, a multi-head self-attention layer, an adaptive fusion layer, a fully connected layer and a softmax layer. The adversarial domain adaptation training of the domain classifier and the target encoder can reduce the deviation between the target translation and the real language, and the gradient reversal layer can unify the objective functions of the two into a back-propagation process for training; the fine-grained interaction between <attribute, sentence> in the target translation corpus is modeled by using a multi-layer multi-head self-attention layer to obtain an attribute-level sentence representation, and then the final attribute-level vector representation is obtained through an adaptive fusion layer, which is input into the fully connected layer and the softmax layer to obtain the final sentiment prediction result.
[0007] The cross-language attribute-level sentiment classification model based on translation matching is trained by adversarial domain adaptation through a target encoder, a domain classifier, and a gradient reversal layer, so that the distribution deviation between the target translation and the real language after encoding by the target language encoder is reduced. The target language encoder is the BERT-Base model officially provided by Google, which includes a 12-layer Transformer encoder submodule. The target language encoder is used to encode the attribute sequence, the target translation sentence sequence, and the real sentence sequence to obtain A tgt , S tgt , U tgt , and then the sentence sequence representation is passed through the average pooling layer to obtain the corresponding sentence vector representation, as shown in the following formula:
[0008] u tgt =MeanPooling(U tgt )
[0009] s tgt =MeanPooling(S tgt )
[0010] The domain classifier is a two-class classifier. For a sentence vector x, its label z∈{0,1} indicates whether the sample belongs to a translated sentence or a real sentence. One method to maximize the similarity between the two distributions is to maximize the classification loss of the discriminator. The loss function of the domain discriminator is shown in the following formula:
[0011]
[0012] Among them, θ t ∈{θ tgt ,θ d}, θ tgt is the target language encoder parameter, θ d is the source domain discriminator parameter. The gradient reversal layer is used to implement the training method of the encoder and the discriminator against each other, that is, to find the parameter θ d Minimize the loss function and find the parameter θ tgt Maximize the loss function, which is achieved by taking all the parameters θ during back propagation tgt The gradient value is multiplied by a hyperparameter -λ.
[0013] The cross-language attribute-level sentiment classification model based on translation matching performs attribute sentiment classification training through a multi-head self-attention layer and adaptive fusion. The multi-head self-attention layer converts the attribute representation A tgt and the translation sentence representation S tgt After concatenation, it is used as input to model the attribute-level latent vector representation H tgt , the process is shown as follows:
[0014]
[0015] in represents the concatenation operation, and Transformer (·) is composed of X layers of Transformer encoder submodules stacked together. Then, the adaptive fusion layer is used to obtain the attribute-level representation after adaptive fusion, and the adaptive fusion layer is used to obtain the attribute-level representation after adaptive fusion. tgt The attribute sequence representation is concatenated by the average pooling layer to obtain the final vector representation, as shown in the following formula:
[0016]
[0017]
[0018]
[0019] a υ =MeanPooling(A tgt )
[0020]
[0021] Among them, H tgt is the attribute-level latent vector representation, W and b are both trainable parameters, M∈R N×N , N is the maximum length of the concatenation of the attribute sequence and the sentence sequence, M ij represents the attention score of the i-th word with respect to the j-th word in the target translation representation, W t ∈R 1×N is the adaptive weight parameter, α t represents the comprehensive attention score of each position, A tgt Represents an attribute.
[0022] Finally, the final vector representation is input into the fully connected layer and the softmax layer, and the output result is the attribute-specific emotional polarity probability distribution. The emotional category with the largest probability is the prediction result. Therefore, the attribute sentiment classification loss function of the model uses cross entropy, and the overall model loss is expressed as the sum of the domain discriminator loss and the attribute sentiment classification loss, as shown in the following formula:
[0023]
[0024] L=L C (θ tgt )+L D (θ t )
[0025] Where M represents the size of the dataset, θ t ∈{θ tgt ,θ d}, θtgt is the target language encoder parameter, θ d is the source domain discriminator parameter. The model is optimized using the Adam optimizer, and its form is as follows:
[0026]
[0027]
[0028]
[0029] Where β1, β2∈[0,1) are the first-order momentum attenuation coefficient and the second-order momentum attenuation coefficient, respectively, and their values are generally 0.9 and 0.999, and ε is the progressive value, which is generally 1e -ε , represents the parameter gradient matrix in the t-1 iteration; at the tth iteration, X t is the parameter matrix, m t is the first-order momentum, v t is the second-order momentum, which is an intermediate variable, and α is the learning rate.
[0030] Beneficial effects:
[0031] 1) The present invention uses BERT as the target language encoder to make the encoded sequence more meaningful, which is beneficial to improving the attribute sentiment classification performance and reducing the training cost.
[0032] 2) The present invention uses adversarial domain adaptation training of the domain discriminator and the target language encoder, which is beneficial to reducing the distribution deviation between the target translation representation and the real sentence representation caused by machine translation, so as to better utilize the source language with rich corpus resources to improve the performance of attribute-level sentiment classification of the target language with scarce corpus resources.
[0033] 3) The present invention utilizes a multi-head self-attention mechanism to model the fine-grained interaction between attribute representation and target translation representation, and obtains attribute-level sentence vector representation through an adaptive fusion layer, thereby better capturing sentence information and improving attribute sentiment classification performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 is a flow chart of the method of the present invention;
[0035] Figure 2 Schematic diagram of the corpus translation sample used in the present invention;
[0036] Figure 3 Schematic diagram of the Transformer encoder submodule network used in the present invention;
[0037] Figure 4 It is a schematic diagram of the structure of the depth domain adaptation model used in the present invention. DETAILED DESCRIPTION
[0038] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0039] Embodiment: The present invention constructs a cross-language attribute-level sentiment classification method based on translation matching for target language corpus with scarce corpus resources. In the process of model construction, the hyperparameters of the model are set based on the model features, including the number of multi-head self-attention layers, gradient reversal hyperparameters, etc. Figure 1 As shown in the figure, the cross-language attribute level sentiment classification method proposed in the present invention includes domain classification and attribute sentiment classification. Domain classification reduces the Figure 2 The encoded distribution deviation between the target translation and the real language is shown. Attribute sentiment classification enables the model to fully model the attribute-level representation of the corpus, and obtain the attribute-specific sentiment polarity probability distribution through the final representation, and then obtain the sentiment category.
[0040] The domain classification training of the present invention is a kind of adversarial domain adaptation training, such as Figure 1 As shown in the figure, it is mainly implemented through the target language encoder, domain classifier and gradient reversal layer. The process is as follows Figure 4 The target language encoder in the present invention adopts the BERT-Base model officially provided by Google, which includes 12 layers such as Figure 3 The Transformer encoder submodule is shown. The maximum length of the sentence sequence is set to 60, the maximum length of the attribute sequence is 10, the word vector dimension is 768, and the sentence sequence and attribute sequence share the language encoder. The target translation and the unlabeled target language sentence are passed through the encoder to obtain the sequence representation L tgt = tgt , S tgt >、U tgt , where A tgt is the target translation attribute sequence representation, S tgt is the target translation sentence sequence representation, as shown in Formula 1, through the average pooling layer by S tgt and U tgt The target translation sentence vector representation and the unlabeled sentence vector representation are obtained respectively.
[0041]
[0042] The domain classifier is a binary classifier that can distinguish whether the input sentence vector representation is the target translation sentence vector representation or the real sentence vector representation. In order to alleviate the domain shift problem between the translation corpus and the real corpus, the method of maximizing the discriminator classification loss is adopted. Its loss function is shown in Formula 2:
[0043]
[0044] Among them, θ t ∈{θ tgt ,θ d}, θ tgt is the target language encoder parameter, θ d is the source domain discriminator parameter. The gradient reversal layer enables adversarial training of the target language encoder and the domain classifier, that is, to find the parameter θ d Minimize the loss function and find the parameter θ tgt Maximize the loss function, which will transform θ during back propagation tgt The gradient of is multiplied by the hyperparameter -λ to achieve gradient reversal, and the λ value is 5e-7.
[0045] The attribute sentiment classification of the present invention is realized by a multi-head self-attention layer, an adaptive fusion layer, and a final fully connected layer and a Softmax layer, and can be represented by a target translation attribute sequence after the labeled translation language passes through a target language encoder. tgt , target translation sentence sequence representation S tgt Get the final sentiment category. The multi-head self-attention layer uses Figure 3 The Transformer encoder submodule shown in the figure converts A tgt and S tgt After concatenation, the attribute-level sentence representation H of the target translation sample can be obtained. tgt , the process is shown in formula 3.
[0046]
[0047] in represents the concatenation operation, and Transformer (·) is composed of three layers of Transformer encoder submodules stacked together. The final attribute-level vector representation can be obtained through the adaptive fusion layer. First, H needs to be calculated as shown in Formula 4. tgt Attention scores at each position:
[0048]
[0049] Among them, W and b are trainable parameters, M∈R N×N , N is the maximum length of the concatenation of the attribute sequence and the sentence sequence, M ij represents the attention score of the i-th word in the target translation representation with respect to the j-th word. On the one hand, in order to calculate the attention representation of the source language to the target language, the adaptive parameter W is used t Fusion obtains the latent vector representation H tgt The comprehensive attention score of each position in is obtained by the softmax function, and then the attention distribution is combined with the target language latent vector H tgtMultiply them to get the attribute-level representation after adaptive fusion, as shown in Formula 5:
[0050]
[0051] Where W t ∈R 1×N is the adaptive weight parameter, α t Represents the comprehensive attention score of each position. Finally, with A tgt The attribute sequence representation is concatenated by the average pooling layer to obtain the final vector representation. The process is shown in Equation 6:
[0052]
[0053] Finally, the final vector representation r is input into the fully connected layer and the softmax layer, and the output result is the attribute-specific sentiment polarity probability distribution. The sentiment category with the highest probability is the prediction result, as shown in Formula 7:
[0054] y=softmax(Wr+b) (7)
[0055] Where W is the weight matrix of the fully connected layer and b is the bias term. After obtaining the conditional probability distribution of each emotion category, the loss function shown in Formula 8 is constructed as the training target:
[0056]
[0057] Where M represents the size of the dataset, θ tgt is the target language encoder parameter, y is the probability that the sample is a positive example in the probability distribution, is the actual category label. If the sample is a positive example, the value is 1, otherwise it is 0. When the model is trained as a whole, as shown in Formula 9, the sum of the domain classification loss function and the attribute sentiment classification loss function is selected as the optimization target.
[0058] L=L C (θ tgt )+L D (θ t ) (9)
[0059] In order to verify the advantages of the present invention in attribute-level sentiment classification of translation corpus, a series of comparative experiments were conducted, using accuracy and F1 value as experimental evaluation criteria. The accuracy calculation method is shown in Formula 10, and the F1 value calculation method is shown in Formula 11.
[0060]
[0061]
[0062] Where Correct represents the number of samples with correct predictions, N represents the total number of samples, R represents the average recall rate of all categories, and P represents the average precision rate of all categories. For any category k, precision represents the proportion of samples predicted correctly by all models as category k; recall represents the proportion of samples predicted correctly as category k, and its calculation method is shown in Formula 12.
[0063]
[0064] The experimental steps mainly include three aspects: first, data preparation; then model training; and finally, attribute-level sentiment classification is performed through the trained model to demonstrate subjective and objective effects.
[0065] 1) Data preparation
[0066] The datasets used in the experiment are attribute-level annotated corpora from SemEval-2016 and public corpora from Suning.com e-commerce platform. The former contains attribute-level annotated corpora in English, Spanish and Russian, while the latter provides Chinese corpora. Based on the annotation specifications of the SemEval-2016 notebook domain dataset and the characteristics of the Chinese language, attribute-level annotation specifications suitable for Chinese corpora were formulated for Suning.com corpora. The specific annotation samples are shown in Table 1. In order to comprehensively evaluate the performance of the cross-language attribute sentiment classification model, the corpus contains review texts from two fields, namely the catering field and the notebook field. In the catering field dataset, English is used as the source language, and Spanish and Russian are used as the target languages for cross-language attribute-level sentiment classification research; in the notebook field dataset, English is used as the source language and Chinese is used as the target language for research. The detailed corpus distribution statistics are shown in Table 2.
[0067] Table 1 Example of attribute-level sentiment analysis corpus annotation
[0068]
[0069]
[0070] Table 2 Distribution statistics of the data sets used in the experiment
[0071]
[0072] During the experiment, both the training set and the development set selected the source language corpus or its target translation version, without using any additional annotated target language corpus resources. The ratio of training set to development set was 8:2, and the test set selected the target language corpus with attribute annotations for testing.
[0073] 2) Model training
[0074] The model is optimized using the Adam optimizer, and the initial learning rate is set to 1e-5 for training the attribute sentiment classifier. In order to achieve adversarial training, the initial learning rate of the domain discriminator is set to 1e-5. In addition, the batch size of the model training is 32, the number of training iterations is 10, and the random inactivation rate of neurons is set to 0.3 to reduce the impact of overfitting. In order to verify the effectiveness of the cross-language attribute-level sentiment classification method based on translation matching proposed in this invention, the following models are selected for comparative experiments:
[0075] MT-DAN: This method translates the source language into the target language. Based on the target translation corpus, a deep average network is trained using the translated corpus to generate sentence vector representations for sentiment classification. Finally, a target language sentiment classification model is constructed and directly tested with the real target language corpus during the test phase.
[0076] ATAE-LSTM (S2T): This method translates the source language into the target language. Based on the target translation corpus, the LSTM network is used to model attribute representation and sentence representation, and the attribute representation is added to the sentence representation using the attention mechanism to generate attribute-level sentence representation for sentiment classification. Finally, a target language sentiment classification model is constructed, and the real target language corpus is directly used for testing in the test phase.
[0077] ATAE-LSTM (T2S): This method uses LSTM network to model attribute representation and sentence representation based on source language corpus, and uses attention mechanism to generate attribute-level sentence representation for sentiment classification. Finally, a source language sentiment classifier is constructed, and in the test phase, the real corpus is translated into the source language using machine translation tools for testing.
[0078] BERT (S2T): This method translates the source language into the target language. Based on the target translation corpus, it constructs <attribute, sentence> pairs as model input to model attribute-level sentence representations. The vector representation obtained by the average pooling layer is used for sentiment classification. Finally, a target language sentiment classification model is constructed, and the real target language corpus is directly used for testing in the test phase.
[0079] BERT (T2S): This method builds <attribute, sentence> pairs as model input to model attribute-level sentence representation based on source language corpus. The vector representation obtained by the average pooling layer is used for sentiment classification. Finally, a source language sentiment classifier is built. In the test phase, the real corpus is translated into the source language using machine translation tools for testing.
[0080] mBERT: This method builds <attribute, sentence> pairs as model input to model attribute-level sentence representation based on the source language corpus. The vector representation obtained by the average pooling layer is used for attribute sentiment classification. Using the cross-language representation capability of mBERT, the real target language corpus is directly used for testing in the test phase.
[0081] mBERT-adv: This method is based on the source language corpus and the unlabeled target language corpus. It introduces a language discriminator and mBERT for adversarial training to generate language-independent feature representations. At the same time, it uses the labeled source language corpus for attribute-level sentiment classification training. In the test phase, the real target language corpus is used for testing.
[0082] TransMatch: This method translates the source language into the target language. Based on the target translation corpus and the unlabeled real target language corpus, the target language encoder is used to encode the attributes, target translation sentences and real sentence representations respectively. Then, the domain discriminator is introduced to conduct adversarial training with the encoder to generate domain-independent feature representations. At the same time, the target translation corpus is used for attribute-level sentiment classification training. In the test phase, the real target language corpus is directly used for testing.
[0083] 3) Experimental results
[0084] The above model is trained and tested using the prepared data set, and the results shown in Table 3 are obtained. The results show the accuracy and F1 value of the trained model on the test set. The larger the values of the two, the better the performance of the model.
[0085] Table 3 Comparison of experimental results of various attribute-level sentiment classification methods
[0086]
[0087]
[0088] From the results in Table 3, it can be seen that the performance of the model proposed by the present invention is optimal. From the experimental results of ATAE-LSTM (S2T) and MT-DAN, it can be seen that the fine-grained interaction between modeling attributes and sentence representation is of great significance. From the two groups of results of ATAE-LSTM (S2T), ATAE-LSTM (T2S) and BERT (S2T), BERT (T2S), it can be found that the effect of forward translation is worse than that of reverse translation. The main reason is that the forward translation method uses machine translation tools to translate the source language corpus into the target language corpus, but the quality of the translation result is low, which makes the performance of the model trained based on the target translation corpus poor, that is, error propagation; while the reverse translation method trains a source language sentiment classifier based on the source language corpus, and translates the target language into the source language in the test phase. The source language corpus used by the classifier is of high quality and has good performance. Even if the translation corpus of poor quality is used in the test phase, it can still maintain good performance.
[0089] Furthermore, by comparing mBERT and mBERT-adv, we can find that introducing a language discriminator during the training process can improve the performance of cross-language attribute sentiment classification. mBERT achieves good performance without the help of any external corpus or machine translation tools, which also illustrates the potential of current cross-language pre-training models.
[0090] Finally, the performance of the TransMatch method proposed in the present invention is better than all other benchmark models, and its accuracy and F1 value are significantly better than mBERT-adv. The reason for this is that the mBERT-adv method introduces a language discriminator in the training stage without the help of machine translation tools, aiming to reduce the distribution difference problem between two different languages, but this method cannot directly improve the cross-language representation ability of mBERT, and the actual effect is not ideal. On the contrary, TransMath first migrates the corpus with the help of machine translation tools. This process has eliminated most of the language differences, but there is still a domain bias between the translation results and the real corpus. Therefore, the domain discriminator is introduced to eliminate the distribution bias, thereby achieving better performance.
[0091] It should be noted that the above embodiments are merely preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Equivalent replacements or substitutions made on the basis of the above technical solutions all fall within the protection scope of the present invention.
Claims
1. A cross-language attribute-level sentiment classification method based on translation matching, It is characterized in that The method includes two parts: domain classification and attribute sentiment classification. In the domain classification part, adversarial training between a domain discriminator and a language encoder is used to reduce the domain deviation problem between the real language and the translated language caused by machine translation. In the attribute sentiment classification part, the encoded attribute sequence and the target translation sentence sequence are input into the network to obtain the final sentiment prediction result. The attribute sentiment classification part includes a target language encoder, a multi-head attention layer, an adaptive fusion layer, a fully connected layer and a softmax layer. The target language encoder uses the multi-head attention layer and the adaptive fusion layer to obtain the attribute sequence representation and the target translation sentence sequence representation to obtain the attribute-level sentence representation of the fine-grained interaction between the attribute and the sentence, and then uses the fully connected layer and the softmax layer to obtain the final sentiment prediction result. The adaptive fusion layer first uses the attention additive model to calculate the attention score of each position of the attribute-level latent vector representation, and then uses the adaptive parameters and the sotfmax function to obtain the comprehensive attention distribution. Finally, the attention distribution is multiplied by the target language latent vector representation to obtain the attribute-level representation after adaptive fusion, and the result of the attribute sequence representation after the average pooling layer is spliced to obtain the final vector representation. The process is shown in the following formula: a υ =MeanPooling(A tgt ) Among them, H tgt is the attribute-level latent vector representation, W and b are both trainable parameters, M∈R N×N , N is the maximum length of the concatenation of the attribute sequence and the sentence sequence, M ij represents the attention score of the i-th word with respect to the j-th word in the target translation representation, W t ∈R 1×N is the adaptive weight parameter, α t represents the comprehensive attention score of each position, A tgt For attribute representation, The loss function during model training is the sum of the domain classification loss function and the attribute sentiment classification loss function. The model is optimized using the Adam optimizer, and its form is as follows: where β 1 , β 2 ∈[0,1), are the first-order momentum attenuation coefficient and the second-order momentum attenuation coefficient, with values of 0.9 and 0.999, respectively, and ε is the progressive value, with a value of 1e -ε , represents the parameter gradient matrix in the t-1 iteration; at the tth iteration, X t is the parameter matrix, m t is the first-order momentum, v t is the second-order momentum, which is an intermediate variable, and α is the learning rate.
2. According to the method for cross-language attribute-level sentiment classification based on translation matching in claim 1, It is characterized in that The domain classification part is a binary classification problem, including a target language encoder, an average pooling layer, a gradient reversal layer, a fully connected layer and a softmax layer. The target language encoder encodes the labeled translation text and the unlabeled target text to obtain an attribute sequence representation A. tgt , target translation sentence sequence representation S tgt And the real sentence sequence representation U tgt , the sentence sequence representation is input into the average pooling layer to obtain the sentence vector representation, which is then input into the gradient reversal layer, and finally the output result of the gradient reversal layer is input into the fully connected layer and the softmax layer to determine whether the input sentence representation belongs to a translated sentence or a real sentence.
3. A cross-language attribute-level sentiment classification method based on translation matching according to claim 2, It is characterized in that The target language encoder is the BERT-Base model officially provided by Google, which includes a 12-layer Transformer encoder submodule.
4. A cross-language attribute-level sentiment classification method based on translation matching according to claim 2, It is characterized in that The loss function of the domain classification part is shown as follows: Among them, θ t ∈{θ tgt ,θ d }, θ tgt is the target language encoder parameter, θ d is the source domain discriminator parameter. The gradient reversal layer is used to implement the training method of the encoder and the discriminator against each other, that is, to find the parameter θ d Minimize the loss function and find the parameter θ tgt Maximize the loss function, which is achieved by taking all the parameters θ during back propagation tgt The gradient value is multiplied by a hyperparameter -λ, and the value of λ is 5e-7.
5. A cross-language attribute-level sentiment classification method based on translation matching according to claim 4, It is characterized in that The multi-head self-attention layer uses the Transformer encoder submodule internally, which takes the concatenation of attribute representation and translation sentence representation as input. The process is shown in the following formula: in represents the concatenation operation. Transformer (·) is composed of X layers of Transformer encoder submodules stacked together. A tgt is the attribute representation, S tgt H is the translation sentence representation. tgt is the attribute-level latent vector representation.
6. A cross-language attribute-level sentiment classification method based on translation matching according to claim 5, It is characterized in that The loss function of attribute sentiment classification is as follows: Where M represents the size of the dataset, θ tgt is the target language encoder parameter, y is the probability that the sample is a positive example in the probability distribution obtained by the model through the fully connected layer and the softmax layer from the final vector representation r, is the actual category label. If the sample is a positive example, the value is 1, otherwise the value is 0.
