Network model training method and device and computer readable storage medium

Abstract

The invention discloses a network model training method and device and a computer readable storage medium, and the method comprises the steps: sequentially carrying out the self-supervised pre-training, domain data fine tuning and knowledge distillation of a pre-training model, i.e., carrying out the unsupervised pre-training of a super-large-scale neural network model through employing mass data, and carrying out the fine tuning of the pre-training model through employing a limited labeled sample, and compressing the fine-tuned super-large model into a target model by using a knowledge distillation method, so as to meet the deployment requirements of target equipment. On the basis, the dependence on annotation data can be reduced, the manual annotation cost can be reduced, the problem that the manual annotation data cost is high can be solved, the universality and generalization of the model can be improved, and the target task precision of the output target model exceeds that of an original customized model.

Description

technical field [0001] Embodiments of the present invention relate to but are not limited to the technical field of deep learning, and in particular, relate to a network model training method, device, and computer-readable storage medium. Background technique [0002] At present, artificial intelligence (AI) technology is centered on machine learning, especially deep learning. It has developed rapidly in application fields such as computer vision, speech, and natural language, and has begun to empower various industries. Of course, there are many reasons for this. A very common defect in the implementation of industrial applications is that the generality and generalization of the model are relatively poor. For a problem in a specific field, In the past, it was generally necessary to collect data, manually label, design models, train models, and repeatedly adjust parameters to finally output a usable model. The development cycle was long, and when faced with a problem in a n...

AI Technical Summary

Problems solved by technology

[0004] Embodiments of the present invention provide a network model training method, device, and computer-readable storage medium, which can solve the problem of high cost of manual labeling data, and can improve the versatility and generalization of the model

Images