Marketing activity prediction model structure and prediction method based on knowledge distillation

Abstract

The invention discloses a marketing activity prediction method based on knowledge distillation. The method comprises the steps of data preprocessing, data set division and network training framework formation of a teacher model, data set division and network training framework formation of a student model, prediction model establishment, marketing activity prediction and the like. The method includes following steps: firstly, constructing a relatively complex teacher model Net-T with a residual neural network as a core, and then establishing a student model Net-S formed by a simple neural network; weighting a soft label obtained by training a teacher model Net-T at a high temperature and a hard label obtained by training a student model Net-S at the same temperature to obtain a total loss function of knowledge distillation; by taking the total loss function as a target function of a student model Net-S during actual deployment, training to obtain a final neural network model, and performing prediction, wherein the result shows that the hybrid model effectively expands the application of deep learning to advertisement calculation and recommendation system algorithms, so that the method significantly improves the accuracy of user click prediction.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence in Internet marketing, and more specifically, to a knowledge distillation-based marketing activity prediction model structure and prediction method. Background technique [0002] With the rapid development of deep learning algorithms and their successful applications in many fields, for example, in the field of Computer Vision (CV), the residual neural network (ResNet) is used to better solve the problem of gradient disappearance in the training process. In the field of Natural Language Processing (NLP), the Transformer model and the Bert model have achieved super processing capabilities for text data. The above-mentioned revolutionary technology has rapidly improved the application effect of deep learning algorithms in different fields, and has also accelerated its implementation. However, as the training data increases, the network model becomes more and more complex, an...

AI Technical Summary

Problems solved by technology

[0004] ①. The actual business volume faced by computing advertising and recommendation systems is often very large, and recommendation systems often need to have strong timeliness

This means that although the offline test of the deep learning model can rely on hardware (such as GPU acceleration), for online deployment, if the deep learning model is too complex and has too many parameters, the corresponding speed of the model will be too high. Slow, especially for specific business scenarios when the traffic is large, it will not be able to meet the needs of timely push

[0005] ② Under normal circumstances, the offline training model and the online deployment model are not deliberately distinguished, that is, the model with better offline training effect is generally directly moved to the online for corresponding deployment; however, those skilled in the art Clearly, there is some inconsistency between the trained model and the deployed model

In addition, large models have high requirements for deployment resources (memory and video memory, etc.); at the same time, we have strict restrictions on delay and computing resources during deployment; therefore, large models are generally inconvenient to deploy directly to services

[0006] ③. When the recommendation system works on the actual line, it may face the adjustment and change of data structures such as features, and the complex large model is generally less flexible than the small model in terms of adjustment and other aspects, adding additional calculation overhead

Images