Language recognition method based on language model and text classification method and device

Abstract

The invention relates to a language recognition method and device based on a language model, a text classification method and device, computer equipment and a storage medium. The method comprises thesteps of obtaining a training word vector corresponding to a training statement, and inputting the training word vector into a to-be-trained first model and a trained second model to obtain a featurematrix output by each first network layer of the first model and a feature matrix output by each second network layer of the second model, wherein the first network layers and the second network layers are in one-to-one correspondence, and the network layer number of the first model is smaller than that of the second model; performing similarity calculation on the feature matrix output by each first network layer and the feature matrix output by the second network layer corresponding to each first network layer; and obtaining each similarity, adjusting model parameters of the first model basedon each similarity until the updated target similarity meets a convergence condition, obtaining a trained first model, and performing language recognition through the first model. The method can improve the model training efficiency.

Description

technical field [0001] The present application relates to the technical field of the Internet, in particular to a language model-based language recognition method, a text classification method, a device, a computer device and a storage medium. Background technique [0002] With the development of Internet technology, neural network models have emerged, which can be used to handle various NLP (Natural Language Processing, natural language processing) tasks. Large neural network models can be compressed into small neural network models through knowledge distillation. [0003] In traditional technologies, the knowledge distillation method for large neural network models is usually based on specific NLP tasks, and small neural network models are obtained by performing knowledge distillation on large neural network models. For example, based on the text similarity matching task definition for large The neural network model knowledge distillation obtains the loss function of the ...

AI Technical Summary

Problems solved by technology

Due to the complex parameters of the large neural network model, it will take more time to re-distill knowledge, resulting in low model training efficiency

Images