Natural language processing model training method, task execution method, device and system

Abstract

The invention discloses a natural language processing model training method, natural language processing method, equipment and system, belonging to the field of natural language processing, including: using marked original data sets to train the teacher model; Enhanced to obtain enhanced text sentences, and use the trained teacher model to mark the enhanced text sentences to obtain the marked enhanced data set; use the original data set and enhanced data set as the training data set, and carry out the student model Training, using the trained student model as a natural language processing model; where both the teacher model and the student model are deep learning models, and perform the same natural language processing tasks, the teacher model is more complex and larger in scale. The present invention can effectively enhance the data set of the natural language processing task in the knowledge distillation scene, improve the processing capability of the natural language processing model, thereby improving the execution effect of the natural language processing task.

Description

technical field [0001] The invention belongs to the field of text processing, and more specifically, relates to a natural language processing method and system in a knowledge distillation scenario. Background technique [0002] Deep learning has been widely used in the field of natural language processing in recent years. The essence of deep learning is to learn more useful features by building machine learning models with many hidden layers and massive training data, so as to ultimately improve classification or prediction. accuracy. When using a large-scale data set to train a deep learning model, in order to deal with complex data distributions, one approach is to build a complex neural network model, such as a residual network with hundreds of layers. This complex network often contains millions of Another method is to mix multiple models, train several large-scale neural networks on the same data set, and then integrate multiple models to obtain the final classificatio...

AI Technical Summary

Problems solved by technology

In general, due to the lack of effective data enhancement methods, the natural language processing ability of the student model trained by knowledge distillation needs to be improved

Images