Natural language processing model training method, task execution method, equipment and system

Abstract

The invention discloses a natural language processing model training method, a natural language processing method, natural language processing equipment and a natural language processing system, whichbelong to the field of natural language processing, and the method comprises the following steps: training a teacher model by utilizing a marked original data set; enhancing text sentences in the original data set to obtain enhanced text sentences, and labeling the enhanced text sentences by using a trained teacher model to obtain a labeled enhanced data set; taking the original data set and theenhanced data set as a training data set, training the student model, and taking the trained student model as a natural language processing model, wherein the teacher model and the student model are both deep learning models and execute the same natural language processing task, and the teacher model is more complex and larger in scale. According to the invention, the data set of the natural language processing task can be effectively enhanced in a knowledge distillation scene, and the processing capability of the natural language processing model is improved, so that the execution effect of the natural language processing task is improved.

Description

technical field [0001] The invention belongs to the field of text processing, and more specifically, relates to a natural language processing method and system in a knowledge distillation scenario. Background technique [0002] Deep learning has been widely used in the field of natural language processing in recent years. The essence of deep learning is to learn more useful features by building machine learning models with many hidden layers and massive training data, so as to ultimately improve classification or prediction. accuracy. When using a large-scale data set to train a deep learning model, in order to deal with complex data distributions, one approach is to build a complex neural network model, such as a residual network with hundreds of layers. This complex network often contains millions of Another method is to mix multiple models, train several large-scale neural networks on the same data set, and then integrate multiple models to obtain the final classificatio...

AI Technical Summary

Problems solved by technology

In general, due to the lack of effective data enhancement methods, the natural language processing ability of the student model trained by knowledge distillation needs to be improved

Images