Small-sample sentiment classification method based on knowledge distillation of big and small tutors
A sentiment classification and tutor technology, applied in the field of few-sample sentiment classification based on the knowledge distillation of large and small tutors, can solve problems such as practical application obstacles and slow reasoning speed, and achieve the effect of reducing resource consumption, improving accuracy, and reducing distillation time.
Active Publication Date: 2022-07-08
SUZHOU UNIV
View PDF4 Cites 2 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
However, due to the large amount of parameters, expensive computing resources are required to call the model, and the inference speed is also very slow, which hinders practical applications.
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0062] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.
[0063] In the process of model optimization, a large model is often a single complex network or a collection of several networks, which has good performance and generalization ability; while a small model has limited expression ability because of the small network size. Therefore, the knowledge learned by the large model (teacher model) can be used to guide the training of the small model (student model), so that the small model has the same performance as the large model, but the number of parameters is greatly reduced, so as to achieve model compression and acceleration. This process called distillation.
[0064] and figure 2 Compared to the traditional single-teacher and single-st...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention relates to a small-sample sentiment classification method based on knowledge distillation of large and small tutors, which comprises the following steps: collecting unlabeled samples and labeled samples on a large number of sentiment classification tasks, and training a large tutor model and a small tutor model by using the labeled samples; all the unlabeled samples pass through the small tutor model to obtain the uncertainty of the probability of each sample, and then the samples of which the sample probability height is uncertain are screened out according to a threshold value and pass through the large tutor model again; and combining the probability output of the large tutor model and the small tutor model to form a soft label to distill the student model, and performing classification prediction by using the distilled student model. According to the method, the frequency of accessing a great tutor model is reduced, the distillation time in a student model training process is shortened, and the accuracy of classification and identification is improved while resource consumption is reduced.
Description
technical field [0001] The invention relates to the technical field of natural language processing, in particular to a few-sample sentiment classification method based on knowledge distillation of large and small tutors. Background technique [0002] Sentiment classification tasks aim to automatically judge the sentiment polarity (eg, negative and positive) expressed in texts. This task is a research hotspot in the field of natural language processing, and is widely used in many application systems such as opinion mining, information retrieval and question answering systems, and is the basic link of these application systems. Few-sample sentiment classification in sentiment classification means that only a small number of labeled samples can be used when training the classifier. [0003] When performing few-shot sentiment classification, the field of artificial intelligence usually uses machine learning and deep learning algorithms to extract sentiment meaning from a piece ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F40/284G06K9/62G06N3/04G06N3/08
CPCG06F40/284G06N3/08G06N3/047G06N3/048G06N3/045G06F18/2155G06F18/2415
Inventor 李寿山常晓琴周国栋
Owner SUZHOU UNIV