Cross-language unsupervised classification with multi-view transfer learning

Abstract

The invention provides cross-language unsupervised classification with multi-view transfer learning. Embodiments of an unsupervised cross-language sentiment classification model, which may be referred to as a multi-view encoder classifier (MVEC), utilizing an unsupervised machine translation (UMT) system and a language discriminator are presented. Unlike previous Language Model (LM)-based fine tuning methods that adjust parameters based only on classification errors of training data, embodiments employ an encoder-decoder framework of UMT as a regularization component on shared network parameters. In one or more embodiments, a cross-language encoder of an embodiment learns a shared representation, which is effective for reconstructing input sentences of both languages and generating more representative views from the input for classification. Experiments on five language pairs prove that the embodiment of the MVEC is obviously superior to other models for 8 / 11 sentiment classification tasks.

Description

technical field [0001] The present disclosure generally relates to systems and methods for computer learning that can provide improved computer performance, features and usage. More specifically, the present disclosure relates to systems and methods for classification. Background technique [0002] Recent neural network models have achieved remarkable performance in sentiment classification in English and other languages. However, their success depends heavily on the availability of large amounts of labeled data or parallel corpora. In fact, some low-resource languages ​​or applications have limited labeled data, or even no labeled or parallel corpora, which may prevent training a powerful and accurate classifier. [0003] To build classification models (such as sentiment classification models) for low-resource languages, researchers have recently developed cross-lingual text classification (CLTC) models (see XuochenXu and Yiming Yang, “Cross-lingual distillation for text...

AI Technical Summary

Problems solved by technology

However, under the "zero parallel resources" setting, encoders trained from self-supervised masked language modeling within each language may not capture semantic similarities between languages ​​well, which may compromise the generality of the fine-tuned model. performance

Images