Text processing model training method and text processing method and device

Abstract

The invention discloses a text processing model training method, a text method and a device in the field of natural language processing in the field of artificial intelligence. The training method comprises the steps of obtaining a training text; respectively inputting the training text into a teacher model and a student model to obtain sample data output by the teacher model and prediction data output by the student model, the teacher model and the student model respectively comprising an input layer, one or more intermediate layers and an output layer, wherein the sample data comprises sample semantic features output by a middle layer of the teacher model and sample labels output by an output layer, and the prediction data comprises prediction semantic features output by the middle layerof the student model and prediction labels output by the output layer; and training model parameters of the student model based on the sample data and the prediction data to obtain a target student model. According to the technical scheme, knowledge migration is effectively carried out on the student model, so that the accuracy of the text processing result of the student model is improved.

Description

technical field [0001] The present application relates to the field of natural language processing, and more specifically, to a text processing model training method, text processing method and device. Background technique [0002] Artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is the branch of computer science that attempts to understand the nature of intelligence and produce a new class of intelligent machines that respond in ways similar to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making. [0003] With the continuo...

AI Technical Summary

Problems solved by technology

[0004] At present, smart terminals usually have requirements such as model size and running time for the deployed models. The existing pre-trained language model has a large number of parameters and a long inference time, which is difficult to meet the deployment requirements of smart terminals.

Images