Model training method and device, computer equipment and storage medium

Abstract

The invention relates to a model training method and device, computer equipment, a storage medium and a computer program product. The method comprises the steps of inputting text sample data into a first model, and determining first loss according to obtained first sample feature data; inputting the text sample data into a second model, and determining a second loss according to the obtained second sample feature data and the first sample feature data; inputting the text sample data into a third model to obtain third sample feature data, and determining the similarity between the first sample feature data and the third sample feature data based on a preset condition to obtain third loss, or obtaining a third loss according to the similarity between the third sample feature data and the similarity between the first sample feature data and the third sample feature data; and determining a loss function according to the first loss, the second loss and the third loss, wherein the loss function is used for training the first model. According to the scheme, the convergence speed of the first model can be higher, and the text recognition accuracy is higher.

Description

technical field [0001] The present application relates to the technical field of natural language processing, and in particular, to a model training method, apparatus, computer equipment, storage medium and computer program product. Background technique [0002] With the development of deep learning, the use of deep neural networks in natural language processing is increasing. In order to improve the performance of the model, most models are relatively complex, with a large number of parameters and large memory consumption, which are difficult to directly apply to the problem. On devices with limited application resources such as GPU (graphics processing unit, graphics processor) and smart phones. [0003] Knowledge distillation belongs to a transfer learning method, which is to transfer the performance of one model to another model. For teacher-student network, the teacher network is often a more complex network with better performance and generalization ability. The teach...

AI Technical Summary

Problems solved by technology

[0004] Complex models can be compressed to meet operational requirements through knowledge distillation technology, but the accuracy of text recognition by student models obtained through knowledge distillation is often low

Images