Emotion classification continuous learning method based on iterative network combination and storage medium

Abstract

The invention discloses a sentiment classification continuous learning method based on iterative network combination and a storage medium, and the method comprises the steps: linearly combining an original network and a fine tuning network, and carrying out the initialization of weight combination parameters of an intermediate network; retraining the intermediate network based on the original combination parameter, the fine tuning combination parameter and the combination initialization parameter to obtain an optimized weight combination parameter; equivalently converting the intermediate network into a final combined network, wherein the final combined network is used as a new original network when learning a next new task; taking the optimized weight combination parameter as an original combination parameter of the original network of the next new task. According to the continuous learning method based on sentiment classification disclosed by the invention, the problem of catastrophic forgetting of previous knowledge in the sentiment classification continuous learning process of the BERT model is avoided under the condition that the network scale is not increased.

Description

technical field [0001] The present application relates to the technical field of text-based emotion classification, in particular to a continuous learning method and storage medium for emotion classification based on iterative network combination. Background technique [0002] Pre-trained language models, such as GPT, BERT, XLNet, etc., have been proposed and applied to many natural language processing tasks, including sentiment classification tasks. BERT was originally designed to pre-train deep bidirectional representations from unlabeled text, by jointly using the left and right contexts of all layers for prediction. [0003] However, when the BERT model is applied to the continuous learning of emotion classification tasks, there are mainly technical defects in the following aspects: first, the old emotion classification model that has spent a lot of time training is abandoned; second, the old emotion classification model and The data of old tasks needs to be stored cont...

AI Technical Summary

Problems solved by technology

[0003] However, when the BERT model is applied to the continuous learning of emotion classification tasks, there are mainly technical defects in the following aspects: first, the old emotion classification model that has spent a lot of time training is abandoned; second, the old emotion classification model and The data of the old tasks needs to be stored continuously, which takes up a lot of storage resources; 3. If the old sentiment classification model trained before is used again to adapt to the updated data, the model will catastrophically forget the tasks in the old domain

Images