Language model training method and device based on knowledge distillation and text classification method and device

Abstract

The invention relates to a language model training method based on knowledge distillation, a text classification method, a language model training device based on knowledge distillation, a text classification device, electronic equipment and a non-temporary computer readable storage medium. The language model training method based on knowledge distillation comprises a first word vector layer parameter determination step and a language model training step. The text classification method comprises the steps of obtaining a to-be-classified text; based on the to-be-classified text, obtaining a keyword code list of the to-be-classified text through extraction; obtaining a word vector of each keyword corresponding to the to-be-classified text through a language model according to the keyword code list; and through the text classification layer, obtaining a classification result of the to-be-classified text. According to the method, a knowledge distillation method is adopted, the dependence on a labeled sample is reduced while the accuracy of the model is reserved, and the reasoning speed is increased by simplifying the structure of the model, so that the applicability and reliability ofthe text classification method in an intelligent auxiliary secret setting system are improved.

Description

technical field [0001] The disclosure relates to the technical field of data information processing and analysis methods in the security field, and in particular to a language model training method based on knowledge distillation, a text classification method, a language model training device based on knowledge distillation, a text classification device, electronic equipment and non-temporary non-volatile computer readable storage medium. Background technique [0002] Confidentiality work refers to the protection of secret information from being leaked through certain means and preventive measures. It is an important task to maintain information security and an important means to protect the core interests of all aspects of social security from infringement. Classification work is the source and basic work of secrecy work, and doing a good job in classifying work is the premise and basis for doing a good job in secrecy work. With the rapid advancement of informatization con...

AI Technical Summary

Problems solved by technology

The traditional classification work relies on manual classification and lacks the technical support means of information-based intelligent auxiliary classification decision-making. The classification is highly subjective and prone to mistakes and inaccuracies

However, due to the difficulty in obtaining a large amount of labeled data in the confidential field, the machine learning classification algorithm that requires a large amount of labeled data cannot achieve good results in the confidential field, and the traditional text classification method is also difficult to effectively apply to the confidential field.

Images