Click rate estimation model generation method and device, electronic equipment and storage medium
By generating confidence loss function and optimizing the offline click-through rate prediction model, the problem of unstable effect of the deep neural network click-through rate prediction method and training the model is solved, achieving higher accuracy and stability.
Patent Information
- Application Number
- CN202310786530.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-29
- Publication Date
- 2024-12-31
AI Technical Summary
The existing deep neural network-based click-through rate estimation method has unstable effects when the retraining model is online, and it is impossible to guarantee that the retraining model is more accurate than the model that has been deployed online.
By obtaining the sample's feature vector, click tag and online click rate estimate value, a confidence loss function is generated, and the offline click rate estimate model to be trained based on this loss function is trained and optimized, and the online click rate estimate model is updated, so as to Improve the accuracy of the retraining model.
The optimized retraining model is more accurate than the models that have been deployed online, avoiding the problem of unstable online effects of retraining model and improving the accuracy of click-through rate estimates.
Smart Images
Figure CN119227482A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a method, device, electronic device and storage medium for generating a click-through rate prediction model. Background Art
[0002] Click-through rate estimation, that is, for a given user and product, the estimated probability value of a user clicking on a product. Click-through rate estimation plays an important role in application scenarios such as search engines, online advertising platforms, and recommendation systems. In order to alleviate the difference between training and testing distributions, the following process is usually widely adopted: collect user data, train models based on user data, and then deploy the trained models for online services until the next new retrained model is generated.
[0003] In the related art, the click rate prediction method based on deep neural network usually uses the cross entropy loss function to optimize the model, but this kind of simple empirical risk minimization method cannot guarantee that the iterative retraining model is better than the current online deployed model, which leads to the problem of unstable online effect of the retraining model. In summary, how to train the retraining model to be more accurate than the online deployed model is still an unresolved issue. Summary of the invention
[0004] The present application proposes a method, device, electronic device and storage medium for generating a click rate prediction model.
[0005] The first aspect of the present application proposes a method for generating a click-through rate prediction model, including: obtaining a feature vector, a click label and an online click-through rate prediction value of a sample, wherein the online click-through rate prediction value is the click-through rate prediction value of the sample output by the online click-through rate prediction model; inputting the feature vector into a first offline click-through rate prediction model to be trained to obtain a first offline click-through rate prediction value of the sample; generating a confidence loss function according to the click label, the online click-through rate prediction value and the first offline click-through rate prediction value; performing model training on the first offline click-through rate prediction model to be trained according to the confidence loss function to obtain a trained first offline click-through rate prediction model; and updating the online click-through rate prediction model according to the trained first offline click-through rate prediction model.
[0006] The method for generating a click-through rate prediction model in an embodiment of the present application obtains a feature vector, a click label and an online click-through rate prediction value of a sample, wherein the online click-through rate prediction value is the click-through rate prediction value of the sample output by the online click-through rate prediction model, inputs the feature vector into a first offline click-through rate prediction model to be trained, obtains the first offline click-through rate prediction value of the sample, generates a confidence loss function according to the click label, the online click-through rate prediction value and the first offline click-through rate prediction value, performs model training on the first offline click-through rate prediction model to be trained according to the confidence loss function, obtains a trained first offline click-through rate prediction model, and updates the online click-through rate prediction model according to the trained first offline click-through rate prediction model. The embodiment of the present application generates a confidence loss function based on the click label of the sample, the online click rate estimation value output after the online click rate estimation model, and the first offline click rate estimation value output after the first offline click rate estimation model, and optimizes the first offline click rate estimation model based on the confidence loss function, that is, optimizes the score values of the deployed model and the retrained model to perform click rate estimation, so that the optimized first offline click rate estimation model, that is, the retrained model, has better accuracy than the online click rate estimation model that has been deployed online, avoiding the problem of unstable online effect of the retrained model.
[0007] The second aspect of the present application proposes a device for generating a click-through rate prediction model, including: an acquisition module, configured to acquire a feature vector, a click label and an online click-through rate prediction value of a sample, wherein the online click-through rate prediction value is a click-through rate prediction value of the sample output by the online click-through rate prediction model; an input module, configured to input the feature vector into a first offline click-through rate prediction model to be trained, to obtain a first offline click-through rate prediction value of the sample; a generation module, configured to generate a confidence loss function based on the click label, the online click-through rate prediction value and the first offline click-through rate prediction value; a training module, configured to perform model training on the first offline click-through rate prediction model to be trained according to the confidence loss function, to obtain a trained first offline click-through rate prediction model; and an update module, configured to update the online click-through rate prediction model based on the trained first offline click-through rate prediction model.
[0008] The device for generating a click-through rate prediction model of an embodiment of the present application obtains a feature vector, a click label and an online click-through rate prediction value of a sample, wherein the online click-through rate prediction value is the click-through rate prediction value of the sample output by the online click-through rate prediction model, inputs the feature vector into a first offline click-through rate prediction model to be trained, obtains the first offline click-through rate prediction value of the sample, generates a confidence loss function according to the click label, the online click-through rate prediction value and the first offline click-through rate prediction value, performs model training on the first offline click-through rate prediction model to be trained according to the confidence loss function, obtains a trained first offline click-through rate prediction model, and updates the online click-through rate prediction model according to the trained first offline click-through rate prediction model. The embodiment of the present application generates a confidence loss function based on the click label of the sample, the online click rate estimation value output after the online click rate estimation model, and the first offline click rate estimation value output after the first offline click rate estimation model, and optimizes the first offline click rate estimation model based on the confidence loss function, that is, optimizes the score values of the deployed model and the retrained model to perform click rate estimation, so that the optimized first offline click rate estimation model, that is, the retrained model, has better accuracy than the online click rate estimation model that has been deployed online, avoiding the problem of unstable online effect of the retrained model.
[0009] The third aspect embodiment of the present application proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for generating a click-through rate prediction model as described in the first aspect embodiment above.
[0010] The fourth aspect of the present application provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method for generating a click-through rate prediction model as described in the first aspect of the present application.
[0011] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0013] Figure 1 A schematic diagram of a flow chart of a method for generating a click rate prediction model provided in an embodiment of the present application;
[0014] Figure 2A schematic diagram of a flow chart of a method for generating a click rate prediction model provided in another embodiment of the present application;
[0015] Figure 3 A schematic diagram of a flow chart of a method for generating a click rate prediction model provided in another embodiment of the present application;
[0016] Figure 4 A schematic diagram of the overall process of a method for generating a click rate prediction model provided by another embodiment of the present application;
[0017] Figure 5 A schematic diagram of the structure of a device for generating a click rate prediction model provided in an embodiment of the present application;
[0018] Figure 6 A block diagram of an electronic device provided in accordance with an embodiment of the present application. DETAILED DESCRIPTION
[0019] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0020] Data processing (DP) is the collection, storage, retrieval, processing, transformation and transmission of data. The basic purpose of data processing is to extract and derive valuable and meaningful data for certain people from a large amount of data that may be disorganized and difficult to understand.
[0021] The following describes a method, device, electronic device, and storage medium for generating a click rate prediction model according to an embodiment of the present application with reference to the accompanying drawings.
[0022] Figure 1 A flowchart of a method for generating a click rate prediction model provided in one embodiment of the present application.
[0023] like Figure 1 As shown, the method for generating a click rate prediction model in the embodiment of the present application may specifically include the following steps:
[0024] S101, obtaining a feature vector, a click label, and an online click rate estimate of a sample, where the online click rate estimate is a click rate estimate of the sample output by an online click rate estimation model.
[0025] Specifically, the execution subject of the method for generating a click rate prediction model in the embodiment of the present application may be the generation device of the click rate prediction model provided in the embodiment of the present application. The generation device of the click rate prediction model may be a hardware device with data information processing capabilities and / or the necessary software required to drive the hardware device to work. Optionally, the execution subject may include a workstation, a server, a computer, a user terminal and other devices. Among them, the user terminal includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a car terminal, etc.
[0026] In an embodiment of the present application, the online click-through rate prediction model is a click-through rate prediction model deployed on an online service platform. The sample is the user and product that serve as the input of the online click-through rate prediction model. The feature vector of the sample is the feature vector generated based on the relevant data of the sample. The relevant data of the sample may specifically include but is not limited to user identification (Identity Document, referred to as ID), product ID, user behavior sequence, product category ID, product brand ID and other related information. Click label means whether the product is clicked by the user in the actual application. The online click-through rate prediction value is the click-through rate prediction value output by the online click-through rate prediction model by inputting the feature vector of the sample into the online click-through rate prediction model. This step S101 is used to obtain training data for offline training of the first offline click-through rate prediction model.
[0027] S102: Input the feature vector into a first offline click rate estimation model to be trained to obtain a first offline click rate estimation value of the sample.
[0028] Specifically, the feature vector of the sample obtained in step S101 is input into the first offline CTR prediction model to be trained, and the CTR estimation value of the sample output by the first offline CTR prediction model to be trained is recorded as the first offline CTR estimation value.
[0029] S103, generating a confidence loss function according to the click label, the online click rate estimate and the first offline click rate estimate.
[0030] Specifically, based on the click label and online click rate estimate of the sample obtained in step S101, and the first offline click rate estimate obtained in step S102, the loss function of the first offline click rate prediction model to be trained is calculated. The loss function is not the commonly used cross entropy function, but a confidence loss function.
[0031] It should be noted here that the confidence loss function can be generated based on the click label, the online click rate estimate and the first offline click rate estimate using at least one of the following strictly proper loss functions: hinge loss function, logistic loss function and mean square error loss function, etc.
[0032] The confidence loss function may specifically include at least one of the following loss functions: a confidence loss function based on a single sample and a confidence loss function based on positive and negative samples of the same user. Positive samples are products that are clicked by users among the exposed products, and negative samples are products that are not clicked by users among the exposed products.
[0033] 1) Confidence loss function based on a single sample.
[0034] The optimization goal of the loss function (i.e., the optimization goal when optimizing the model based on the loss function) is to maximize the estimated value of the sample output by the model, so that it is better than the estimated value of the online model, that is, to maximize the difference between the model output and the estimated value, so that the training model (i.e., the first offline click-through rate prediction model) has better accuracy than the online service model (i.e., the online click-through rate prediction model), that is, to maximize the difference between the output and the estimated value of the positive sample, and to minimize the difference between the output and the estimated value of the negative sample, that is, for the positive sample, the optimization goal of the loss function is to maximize the difference between the first offline click-through rate estimated value and the online click-through rate estimated value; for the negative sample, the optimization goal of the loss function is to minimize the difference between the first offline click-through rate estimated value and the online click-through rate estimated value.
[0035] The specific form of the loss function is:
[0036]
[0037] Wherein, l(f) is the single-sample confidence loss function of the first offline CTR prediction model f to be trained, is the mathematical expectation, x is the input of the first offline CTR prediction model f to be trained, that is, the feature vector of the sample, y is the click label of the sample, ~ is used to represent sampling, D is the sample used to train the first offline CTR prediction model f, φ is a strictly proper loss function, such as hinge loss, logistic loss, and mean square error loss, etc. Any strictly proper loss function here is suitable for calculating the confidence loss function based on a single sample, f(x) is the first offline CTR estimate value output by the first offline CTR prediction model f to be trained after normalization, f online (x) is the online click rate estimation value output by the normalized online click rate estimation model f.
[0038] Taking φ as the hinge loss function as an example, the single-sample confidence loss function generated by the hinge loss function is:
[0039] l(f)=y*max(0,m–(y_pred–y_pctr))+(1-y)*max(0,m–(y_pctr–y_pred));
[0040] Among them, l(f) is the single-sample-based confidence loss function of the first offline click-through rate prediction model f to be trained, y is the click label, max is the maximum value function, m is the preset confidence threshold, m is greater than or equal to 0 and less than or equal to 1, y_pred is the first offline click-through rate prediction value, and y_pctr is the online click-through rate prediction value.
[0041] 2) Confidence loss function based on positive and negative samples of the same user.
[0042] Usually, the click-through rate model pays more attention to the ranking index, namely the area under the receiver operating characteristic (ROC) curve (AUC), that is, the ratio of the number of positive samples before negative samples in the total positive and negative samples. This loss function aims to relatively improve the ranking performance of the training model. The optimization goal of this loss function (that is, the optimization goal when optimizing the model based on this loss function) is to maximize the estimated difference of the positive and negative sample pairs under the same user, so that it is larger than the estimated difference of the online model, that is, to maximize the distance between the difference between the positive and negative samples of the training model and the estimated difference of the online model of the sample pair, that is, to maximize the distance between the first difference between the positive and negative samples and the second difference between the positive and negative samples. The first difference between the positive and negative samples is the difference between the first offline click-through rate estimates of the positive and negative samples, and the second difference between the positive and negative samples is the difference between the online click-through rate estimates of the positive and negative samples.
[0043] The specific form of the loss function is:
[0044]
[0045] Wherein, l(f) is the confidence loss function based on positive and negative samples of the first offline CTR prediction model f to be trained, is the mathematical expectation, x is the input of the first offline CTR prediction model f to be trained, that is, the feature vector of the sample, x + is the feature vector of the positive sample, x - is the feature vector of the negative sample, ~ is used to represent sampling, p + is a positive sample, p - is a negative sample, φ is a strictly proper loss function, such as hinge loss, logistic loss, and mean square error loss. Any strictly proper loss function is suitable for calculating the confidence loss function based on positive and negative samples. + ,x - ) is the difference between the first offline click rate estimate of the positive sample and the first offline click rate estimate of the negative sample, that is, the difference between the first offline click rate estimate of the positive and negative samples, d online (x + ,x - ) is the difference between the estimated online click rate of the positive sample and the estimated online click rate of the negative sample, that is, the difference between the estimated online click rate of the positive and negative samples.
[0046] Taking φ as the hinge loss function as an example, the confidence loss function based on positive and negative samples generated by the hinge loss function is:
[0047] l(f)=max(0,m–(y_pred_diff–y_pctr_diff));
[0048] Among them, l(f) is the confidence loss function based on positive and negative samples of the first offline click-through rate prediction model f to be trained, max is the maximum value function, m is the preset confidence threshold, m is greater than or equal to 0 and less than or equal to 1, y_pred_diff is the difference between the first offline click-through rate prediction values of the positive and negative samples, and y_pctr_diff is the difference between the online click-through rate prediction values of the positive and negative samples.
[0049] S104, performing model training on the first offline click rate prediction model to be trained according to the confidence loss function to obtain a trained first offline click rate prediction model.
[0050] Specifically, the first offline click-through rate prediction model to be trained is trained (i.e., model optimization) according to the confidence loss function generated in step S103, and the optimized model is used as the new first offline click-through rate prediction model to be trained to continue the above model training process until the condition for the end of model training is met, such as the loss function value reaches the loss function value threshold, the model training is stopped, and the model after the last optimization is used as the trained first offline click-through rate prediction model.
[0051] S105: Update the online click rate prediction model according to the trained first offline click rate prediction model.
[0052] Specifically, the trained first offline click rate prediction model obtained in step S104 is used as a new online click rate prediction model.
[0053] In summary, the method for generating a click-through rate prediction model in an embodiment of the present application obtains a feature vector, a click label and an online click-through rate prediction value of a sample, where the online click-through rate prediction value is the click-through rate prediction value of the sample output by the online click-through rate prediction model, inputs the feature vector into the first offline click-through rate prediction model to be trained, obtains the first offline click-through rate prediction value of the sample, generates a confidence loss function according to the click label, the online click-through rate prediction value and the first offline click-through rate prediction value, performs model training on the first offline click-through rate prediction model to be trained according to the confidence loss function, obtains a trained first offline click-through rate prediction model, and updates the online click-through rate prediction model according to the trained first offline click-through rate prediction model. The embodiment of the present application generates a confidence loss function according to the click label of the sample, the online click rate estimation value output after the online click rate estimation model, and the first offline click rate estimation value output after the first offline click rate estimation model, optimizes the first offline click rate estimation model based on the confidence loss function, and uses the optimized first offline click rate estimation model as the new online click rate estimation model, that is, the optimized first offline click rate estimation model is used as a retraining model, which is optimized based on the deployed online click rate estimation model. Therefore, the optimized first offline click rate estimation model is more accurate than the online click rate estimation model that has been deployed online, avoiding the problem of unstable online effect of the optimized first offline click rate estimation model as a retraining model.
[0054] Figure 2 A flow chart of a method for generating a click rate prediction model provided by another embodiment of the present application. Figure 2 As shown in the above Figure 1 Based on the illustrated embodiment, the method for generating a click rate prediction model in the embodiment of the present application may specifically include the following steps:
[0055] S201, obtaining a feature vector, a click label, and an online click rate estimate of a sample, where the online click rate estimate is a click rate estimate of the sample output by an online click rate estimation model.
[0056] S202: Input the feature vector into a first offline click rate estimation model to be trained to obtain a first offline click rate estimation value of the sample.
[0057] S203, generating a confidence loss function according to the click label, the online click rate estimate, and the first offline click rate estimate.
[0058] Specifically, steps S201-S203 in this embodiment are the same as steps S101-S103 in the above embodiment, and are not described again here.
[0059] In the above embodiment, step S104 of "performing model training on the first offline click rate prediction model to be trained according to the confidence loss function to obtain a trained first offline click rate prediction model" may specifically include the following step S204.
[0060] S204, performing model training on the first offline click rate prediction model to be trained by a back propagation algorithm according to the confidence loss function to obtain a trained first offline click rate prediction model.
[0061] Specifically, according to the confidence loss function generated in step S203, the first offline click-through rate prediction model to be trained is optimized through a back propagation algorithm, such as a stochastic gradient descent (SGD) algorithm and an adaptive momentum gradient descent (Adam) method, to obtain a trained first offline click-through rate prediction model.
[0062] S205: Update the online click rate prediction model according to the trained first offline click rate prediction model.
[0063] Specifically, step S205 in this embodiment is the same as step S105 in the above embodiment, and will not be described in detail here.
[0064] It should be noted here that if Figure 3 As shown, the "obtaining a feature vector of a sample" in the above step S201 may specifically include the following steps:
[0065] S301, performing feature processing on sample data of a sample to obtain features of the sample.
[0066] Specifically, feature processing and discretization may be performed on sample data to obtain sample features.
[0067] S302, performing encoding vectorization processing on the features to obtain the encoded feature vector of the sample.
[0068] Specifically, one-hot encoding can be used to vectorize the features corresponding to the user sparse ID and product sparse ID in the sample data, and one-hot or multi-hot encoding can be used to vectorize other sparse IDs and user behavior sequences to obtain the encoded feature vector of the sample.
[0069] S303, extracting the representation embedding layer features of the encoded feature vector to obtain a feature vector.
[0070] Specifically, the high-order sparse vector is used to extract the representation embedding layer features of the encoded feature vector obtained in step S302 to obtain a feature vector of the sample.
[0071] As a feasible implementation method, the trained first offline click-through rate prediction model uses samples of the first time granularity for training, the initial model of the first offline click-through rate prediction model to be trained is the trained second offline click-through rate prediction model, and the second offline click-through rate prediction model is trained using samples of the second time granularity, and the second time granularity is larger than the first time granularity.
[0072] Specifically, the training data of the sample obtained in step S201 is processed into training data of the first time granularity, such as minute level. At this stage, the system will perform real-time training of the first offline click-through rate prediction model. In the real-time training, the second offline click-through rate prediction model trained based on the training data of the second time granularity, such as day level, is preloaded as the initial model of the first offline click-through rate prediction model to be trained. Subsequently, the minute level training data is input into the first offline click-through rate prediction model to be trained. After the confidence loss function is calculated, the model is trained according to the confidence loss function. After each training is completed, the model is exported and pushed to the online service platform as a new online click-through rate prediction model. After the new second offline click-through rate prediction model at the day level is updated, the new second offline click-through rate prediction model at the day level is reloaded as the initial model of the first offline click-through rate prediction model to be trained.
[0073] The training process of the second offline CTR prediction model at the second time granularity, such as the day level, is as follows: the minute-level training data is used to be recombined into the day-level training data according to the user-query, and then these data are sequentially input into the second offline CTR prediction model to be trained, and after the confidence loss function is calculated, the model is trained according to the confidence loss function. After the training is completed, the model is exported as the initial model of the first offline CTR prediction model at the minute level. The initial model of the second offline CTR prediction model to be trained does not require model loading, and all model parameters are initialized directly using the random initialization method.
[0074] In summary, the method for generating a click-through rate prediction model in an embodiment of the present application obtains a feature vector, a click label and an online click-through rate prediction value of a sample, where the online click-through rate prediction value is the click-through rate prediction value of the sample output by the online click-through rate prediction model, inputs the feature vector into the first offline click-through rate prediction model to be trained, obtains the first offline click-through rate prediction value of the sample, generates a confidence loss function according to the click label, the online click-through rate prediction value and the first offline click-through rate prediction value, performs model training on the first offline click-through rate prediction model to be trained according to the confidence loss function, obtains a trained first offline click-through rate prediction model, and updates the online click-through rate prediction model according to the trained first offline click-through rate prediction model. The embodiment of the present application generates a confidence loss function according to the click label of the sample, the online click rate estimation value output after the online click rate estimation model, and the first offline click rate estimation value output after the first offline click rate estimation model, optimizes the first offline click rate estimation model based on the confidence loss function, and uses the optimized first offline click rate estimation model as the new online click rate estimation model, that is, the optimized first offline click rate estimation model is used as a retraining model, which is optimized based on the deployed online click rate estimation model. Therefore, the optimized first offline click rate estimation model is more accurate than the online click rate estimation model that has been deployed online, avoiding the problem of unstable online effect of the optimized first offline click rate estimation model as a retraining model. The initial model of the first offline click-through rate prediction model trained with samples of the first time granularity is the second offline click-through rate prediction model trained with samples of the second time granularity, that is, the initial model of the first offline click-through rate prediction model is a trained model, and training the first offline click-through rate prediction model based on this initial model can enable the first offline click-through rate prediction model to achieve the training effect faster and the performance of the trained first offline click-through rate prediction model is better, that is, the training speed and accuracy of the first offline click-through rate prediction model are improved.
[0075] To clearly illustrate the method for generating the click rate prediction model in the embodiment of the present application, Figure 4 The overall process of the method for generating the click rate prediction model of the embodiment of the present application is described in detail. Figure 4As shown, taking the recommendation system as an example, after the training of the click rate prediction model is completed, it is pushed to the online prediction model system after a certain formatted output and stored in the entire online service platform. When the user requests, the online service platform performs click rate prediction and sorting. In the click rate prediction part, the online service platform needs to collect the sample's feature vector, click label (i.e. click or not clicked) and the online click rate prediction value output by the online click rate prediction model as offline training data, and splice them into minute-level offline data and day-level offline data. Based on the minute-level offline data, the confidence loss function based on the prediction value and the real-time minimum batch offline model training are calculated. Based on the day-level offline data, the confidence loss function based on the prediction value and the offline minimum batch offline model training are calculated. The model trained with the day-level offline data is used as the initial model for model training with the minute-level offline data, and the model trained with the minute-level offline data is pushed back to the online prediction model system.
[0076] In order to implement the above embodiment, the embodiment of the present application also proposes a device for generating a click rate prediction model. Figure 5 FIG. 1 is a schematic diagram of a structure of a device for generating a click rate prediction model according to an embodiment of the present application. Figure 5 As shown, the click rate prediction model generation device 500 of the embodiment of the present application may specifically include: an acquisition module 501, an input module 502, a generation module 503, a training module 504 and an update module 505. Among them:
[0077] The acquisition module 501 is configured to acquire a feature vector, a click label and an online click rate estimate of a sample, where the online click rate estimate is a click rate estimate of the sample output by an online click rate estimation model.
[0078] The input module 502 is configured to input the feature vector into the first offline click rate prediction model to be trained to obtain a first offline click rate prediction value of the sample.
[0079] The generating module 503 is configured to generate a confidence loss function according to the click label, the online click rate estimate and the first offline click rate estimate.
[0080] The training module 504 is configured to perform model training on the first offline click rate prediction model to be trained according to the confidence loss function to obtain a trained first offline click rate prediction model.
[0081] The updating module 505 is configured to update the online click rate prediction model according to the trained first offline click rate prediction model.
[0082] In one embodiment of the present application, the generation module 503 is further configured to generate a confidence loss function based on the click label, the online click rate estimate and the first offline click rate estimate, using at least one of the following loss functions: hinge loss function, logistic loss function and mean square error loss function.
[0083] In one embodiment of the present application, the confidence loss function includes: a confidence loss function based on a single sample and / or a confidence loss function based on positive and negative samples.
[0084] In one embodiment of the present application, for positive samples, the optimization goal of the confidence loss function based on a single sample is to maximize the difference between the first offline click-through rate estimate and the online click-through rate estimate; for negative samples, the optimization goal of the confidence loss function based on a single sample is to minimize the difference between the first offline click-through rate estimate and the online click-through rate estimate.
[0085] In one embodiment of the present application, the confidence loss function includes a confidence loss function based on a single sample; according to the click label, the online click rate estimate and the first offline click rate estimate, the confidence loss function generated by the hinge loss function is: l(f)=y*max(0, m–(y_pred–y_pctr))+(1-y)*max(0, m–(y_pctr–y_pred)); wherein l(f) is the confidence loss function of the first offline click rate prediction model f to be trained, y is the click label, max is the maximum value function, m is a preset confidence threshold, m is greater than or equal to 0 and less than or equal to 1, y_pred is the first offline click rate estimate, and y_pctr is the online click rate estimate.
[0086] In one embodiment of the present application, the optimization goal of the confidence loss function based on positive and negative samples is to maximize the distance between the first difference between the positive and negative samples and the second difference between the positive and negative samples, the first difference between the positive and negative samples is the difference between the first offline click-through rate estimates of the positive and negative samples, and the second difference between the positive and negative samples is the difference between the online click-through rate estimates of the positive and negative samples.
[0087] In one embodiment of the present application, the confidence loss function includes a confidence loss function based on positive and negative samples; according to the click label, the online click rate estimate and the first offline click rate estimate, the confidence loss function generated by the hinge loss function is: l(f)=max(0, m-(y_pred_diff-y_pctr_diff)); wherein l(f) is the confidence loss function of the first offline click rate prediction model f to be trained, max is the maximum value function, m is a preset confidence threshold, m is greater than or equal to 0 and less than or equal to 1, y_pred_diff is the difference between the first offline click rate estimate of the positive and negative samples, and y_pctr_diff is the difference between the online click rate estimate of the positive and negative samples.
[0088] In one embodiment of the present application, the acquisition module 501 is further configured to: perform feature processing on the sample data of the sample to obtain the features of the sample; perform encoding vectorization processing on the features to obtain the encoded feature vector of the sample; perform representation embedding layer feature extraction on the encoded feature vector to obtain a feature vector.
[0089] In one embodiment of the present application, the training module 504 is further configured to: perform model training on the first offline click rate prediction model to be trained through a back propagation algorithm according to a confidence loss function to obtain a trained first offline click rate prediction model.
[0090] In one embodiment of the present application, the trained first offline click-through rate prediction model uses samples of the first time granularity for training, the initial model of the first offline click-through rate prediction model to be trained is the trained second offline click-through rate prediction model, and the second offline click-through rate prediction model is trained using samples of the second time granularity, and the second time granularity is larger than the first time granularity.
[0091] It should be noted that the above explanation of the embodiment of the method for generating a click-through rate prediction model is also applicable to the device for generating a click-through rate prediction model in the embodiment of the present application, and the specific process will not be repeated here.
[0092] In summary, the device for generating a click-through rate prediction model in the embodiment of the present application obtains a feature vector, a click label and an online click-through rate prediction value of a sample, where the online click-through rate prediction value is the click-through rate prediction value of the sample output by the online click-through rate prediction model, inputs the feature vector into the first offline click-through rate prediction model to be trained, obtains the first offline click-through rate prediction value of the sample, generates a confidence loss function according to the click label, the online click-through rate prediction value and the first offline click-through rate prediction value, performs model training on the first offline click-through rate prediction model to be trained according to the confidence loss function, obtains a trained first offline click-through rate prediction model, and updates the online click-through rate prediction model according to the trained first offline click-through rate prediction model. The embodiment of the present application generates a confidence loss function according to the click label of the sample, the online click rate estimation value output after the online click rate estimation model, and the first offline click rate estimation value output after the first offline click rate estimation model, optimizes the first offline click rate estimation model based on the confidence loss function, and uses the optimized first offline click rate estimation model as the new online click rate estimation model, that is, the optimized first offline click rate estimation model is used as a retraining model, which is optimized based on the deployed online click rate estimation model. Therefore, the optimized first offline click rate estimation model is more accurate than the online click rate estimation model that has been deployed online, avoiding the problem of unstable online effect of the optimized first offline click rate estimation model as a retraining model. The initial model of the first offline click-through rate prediction model trained with samples of the first time granularity is the second offline click-through rate prediction model trained with samples of the second time granularity, that is, the initial model of the first offline click-through rate prediction model is a trained model, and training the first offline click-through rate prediction model based on this initial model can enable the first offline click-through rate prediction model to achieve the training effect faster and the performance of the trained first offline click-through rate prediction model is better, that is, the training speed and accuracy of the first offline click-through rate prediction model are improved.
[0093] like Figure 6 , is a block diagram of an electronic device according to a method for generating a click rate prediction model according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as intelligent voice interaction devices, personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.
[0094] like Figure 6As shown, the electronic device includes: one or more processors 601, a memory 602, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor 601 can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 601 is taken as an example.
[0095] The memory 602 is a non-transitory computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable at least one processor to execute the method for generating a click-through rate prediction model provided in the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to execute the method for generating a click-through rate prediction model provided in the present application.
[0096] The memory 602 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the method for generating the click rate prediction model in the embodiment of the present application (for example, the attached Figure 5 The processor 601 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 602, that is, the method for generating the click rate prediction model in the above method embodiment is implemented.
[0097] The memory 602 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created by the use of an electronic device according to the method for generating a click rate prediction model, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 602 may optionally include a memory remotely arranged relative to the processor 601, and these remote memories may be connected to the electronic device of the method for generating a click rate prediction model via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0098] The electronic device of the method for generating a click rate prediction model may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.
[0099] The input device 603 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device of the method for generating the click rate prediction model, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 604 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.
[0100] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0101] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.
[0102] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0103] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0104] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short).
[0105] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.
[0106] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0107] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A method for generating a click rate prediction model, characterized in that: include: Acquire a feature vector, a click label, and an online click rate estimate of the sample, where the online click rate estimate is the click rate estimate of the sample output by an online click rate estimate model; Inputting the feature vector into a first offline click rate prediction model to be trained to obtain a first offline click rate prediction value of the sample; Generate a confidence loss function according to the click label, the online click rate estimate and the first offline click rate estimate; Performing model training on the first offline click rate prediction model to be trained according to the confidence loss function to obtain a trained first offline click rate prediction model; The online click rate prediction model is updated according to the trained first offline click rate prediction model.
2. The generation method according to claim 1, characterized in that: The generating a confidence loss function according to the click label, the online click rate estimate and the first offline click rate estimate includes: According to the click label, the online click rate estimate and the first offline click rate estimate, the confidence loss function is generated using at least one of the following loss functions: hinge loss function, logistic loss function and mean square error loss function.
3. The generation method according to claim 2, characterized in that: The confidence loss function includes: a confidence loss function based on a single sample and / or a confidence loss function based on positive and negative samples.
4. The generation method according to claim 3, characterized in that: For a positive sample, the optimization goal of the single-sample-based confidence loss function is to maximize the difference between the first offline click-through rate estimate and the online click-through rate estimate; For negative samples, the optimization goal of the single-sample-based confidence loss function is to minimize the difference between the first offline click-through rate estimate and the online click-through rate estimate.
5. The generation method according to claim 3, characterized in that: The confidence loss function includes the confidence loss function based on a single sample; According to the click label, the online click rate estimate and the first offline click rate estimate, the confidence loss function generated by using the hinge loss function is: l(f)=y*max(0,m–(y_pred–y_pctr))+(1-y)*max(0,m–(y_pctr–y_pred)); Among them, the is the confidence loss function of the first offline click-through rate prediction model f to be trained, the y is the click label, the max is the maximum value function, the m is a preset confidence threshold, the m is greater than or equal to 0 and less than or equal to 1, the y_pred is the first offline click-through rate prediction value, and the y_pctr is the online click-through rate prediction value.
6. The generation method according to claim 3, characterized in that: The optimization goal of the confidence loss function based on positive and negative samples is to maximize the distance between the first difference between positive and negative samples and the second difference between positive and negative samples, wherein the first difference between positive and negative samples is the difference between the first offline click-through rate estimates of positive and negative samples, and the second difference between positive and negative samples is the difference between the online click-through rate estimates of positive and negative samples.
7. The generation method according to claim 3, characterized in that: The confidence loss function includes the confidence loss function based on positive and negative samples; According to the click label, the online click rate estimate and the first offline click rate estimate, the confidence loss function generated by using the hinge loss function is: l(f)=max(0,m–(y_pred_diff–y_pctr_diff)); Among them, the l(f) is the confidence loss function of the first offline click-through rate prediction model f to be trained, the max is the maximum value function, the m is a preset confidence threshold, the m is greater than or equal to 0 and less than or equal to 1, the y_pred_diff is the difference between the first offline click-through rate prediction values of the positive and negative samples, and the y_pctr_diff is the difference between the online click-through rate prediction values of the positive and negative samples.
8. The generation method according to claim 1, characterized in that: The step of obtaining a feature vector of a sample includes: Performing feature processing on sample data of the sample to obtain features of the sample; Performing encoding vectorization processing on the feature to obtain an encoding feature vector of the sample; Performing embedding layer feature extraction on the encoded feature vector to obtain the feature vector.
9. The generation method according to claim 1, characterized in that: The step of performing model training on the first offline click rate prediction model to be trained according to the confidence loss function to obtain a trained first offline click rate prediction model includes: Model training is performed on the first offline click rate prediction model to be trained according to the confidence loss function through a back propagation algorithm to obtain the trained first offline click rate prediction model.
10. The generation method according to claim 1, characterized in that: The trained first offline click-through rate prediction model uses the samples of the first time granularity for training, the initial model of the first offline click-through rate prediction model to be trained is the trained second offline click-through rate prediction model, the second offline click-through rate prediction model uses the samples of the second time granularity for training, and the second time granularity is larger than the first time granularity.
11. A device for generating a click rate prediction model, characterized in that: include: An acquisition module is configured to acquire a feature vector, a click label, and an online click rate estimate of a sample, wherein the online click rate estimate is a click rate estimate of the sample output by an online click rate estimation model; An input module, configured to input the feature vector into a first offline click rate prediction model to be trained to obtain a first offline click rate prediction value of the sample; A generating module, configured to generate a confidence loss function according to the click label, the online click rate estimate and the first offline click rate estimate; A training module is configured to perform model training on the first offline click rate prediction model to be trained according to the confidence loss function to obtain a trained first offline click rate prediction model; The updating module is configured to update the online click rate prediction model according to the trained first offline click rate prediction model.
12. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.
13. A computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10.