CTR prediction time drift processing method based on Kalman filtering

Abstract

The invention discloses a CTR prediction time drift processing method based on Kalman filtering. The method comprises the steps that prior state estimation and a prior covariance matrix are calculated based on historical states; calculating Kalman gain, and then calculating posterior state estimation and a posterior covariance matrix; designing a new loss function in combination with Kalman loss and CTR prediction loss; and training the existing CTR model based on the posterior state estimation and the new loss function to obtain a new CTR model for processing time drift. According to the CTR prediction time drift processing method based on Kalman filtering disclosed by the invention, comprehensive experiments are carried out on two public data sets, and experimental results show that the method can improve an existing CTR model to obtain a new CTR model, and the new CTR model can effectively process the time drift problem and has better prediction capability.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning and time series analysis, and in particular to a method for processing time drift of CTR prediction based on Kalman filtering. Background Art

[0002] In the field of deep learning, click-through rate (CTR) prediction is a key module. CTR prediction is a fundamental task in online advertising and recommendation systems, aiming to estimate the likelihood that a user will click on a given item or ad. With the introduction of deep learning, CTR prediction has made significant progress. Deep learning can be broadly divided into two categories. The first category focuses on feature interaction patterns, such as factorization machines (FMs), which lay the foundation for capturing feature interactions. Subsequent models such as DeepFM (A Factorization-Machine based Neural Network) and AFM (Attentional Factorization Machines) have further enhanced the ability to model complex feature interactions and improved prediction accuracy. The second category focuses on user behavior, specifically modeling underlying user preferences based on feature interaction sequences. Existing CTR prediction models are primarily based on empirical risk minimization (ERM), which aims to minimize the average loss of training data. While ERM is effective under the assumption that training and test data are independent and identically distributed, this assumption often does not hold in real-world scenarios. In practice, user interests are dynamic, leading to a distribution mismatch between historical training data and real-time test data. This phenomenon of data distribution changing over time is known as temporal drift, which leads to suboptimal performance and degraded user experience. Despite the remarkable effectiveness of existing CTR prediction models, they are still susceptible to the challenges posed by temporal drift. Addressing this challenge is crucial to maintaining the long-term effectiveness of CTR prediction systems.

[0003] Temporal representation augmentation (TLA) is a technique used to improve model performance in time series analysis. It transforms and augments time series data during training to enhance the model's ability to capture temporal dynamics and generalize. The DIL method proposed by Y. Zhang et al. helps CTR prediction models cope with temporal drift by learning context-specific and context-invariant features from feature interactions. The SFTL method proposed by J. Zhu et al. uses fast and slow learners to assist the main network in capturing user interests, thereby enhancing network robustness. While achieving significant results, DIL and SFTL ignore the temporal evolution of historical feature interactions, which helps the model learn intrinsic patterns of temporal drift. Furthermore, the DDA method proposed by Q. Zeng et al. and the CIL method proposed by L. Yong et al. capture temporal evolution in historical image data and achieve significant performance. However, because these methods are designed for image data, they face significant limitations when applied to click data. Therefore, it is imperative to explore solutions that can learn temporal drift evolution trends from historical feature interaction data and adapt to click data. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by providing a Kalman filter-based method for addressing temporal drift in CTR prediction. This method is based on and must be combined with existing CTR prediction models. It utilizes three classic CTR prediction models: DeepFM, Deep & Cross Network (DCN), and Product-based Neural Networks (PNN).

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for processing CTR prediction time drift based on Kalman filtering includes the following steps:

[0007] Step S1: Calculate the prior state estimate and the prior covariance matrix based on the historical state;

[0008] Step S2: Calculate the Kalman gain, and then calculate the posterior state estimate and the posterior covariance matrix;

[0009] Step S3: Combine Kalman loss and CTR prediction loss to design a new loss function;

[0010] Step S4: Train the existing CTR model based on the posterior state estimation and the new loss function to obtain a new CTR model that processes time drift.

[0011] Step S1 calculates the prior state estimate and the prior covariance matrix based on the historical state:

[0012] First, initialize the user interest state vector through the existing CTR model or the user's historical behavior data; give a reasonable estimate based on the observation of the system's initial state or prior knowledge as the initial value of the state estimation ; Based on the historical state, the prior state estimate at the current time step t is predicted, and the current state time step is derived from the posterior state estimate at the previous moment The prior state estimate , by the time step The posterior state estimate of and the state transfer matrix Multiplying them together, the calculation formula is:

[0013]

[0014] In the above expression (1), is the time step The posterior state estimate of is the state transition matrix at time step t.

[0015] Set the non-zero covariance matrix of the initial state Indicates the uncertainty of the initial state estimate; in general, it can be assumed that the initial estimate uncertainty is high and a larger value is chosen is the initial covariance matrix; through the transfer matrix and the process noise covariance matrix , to achieve iterative update of the prior covariance matrix; at time step t, the state transfer matrix The posterior covariance matrix for the previous time step Processing is performed and the process noise covariance matrix is ​​added , get the prior covariance matrix at time step t predicted by the posterior covariance matrix at time step t-1 , the expression is as follows:

[0016]

[0017] In the above expression (2), the state transfer matrix and the process noise covariance matrix are learnable parameters; where the process noise covariance matrix Gaussian noise matrix can be applied to simulate random interference in real environment and enhance the robustness of the model.

[0018] The Kalman gain is calculated as described in step S2, and then the posterior state estimate and the posterior covariance matrix are calculated: the Kalman filter is widely used in time series prediction, wherein the parameters of the classic Kalman filter are incrementally updated during the system state update process.

[0019] Calculate Kalman gain , this gain is used to balance the credibility of the prior estimate and the observed value; in CTR prediction, if the change of user interest leads to the expansion of the prediction error, the prior covariance matrix Increases, then the Kalman gain Increases, the model needs to rely more on observation data to correct the prediction; if the advertising platform has abnormal conditions, the observation noise covariance matrix Increase by reducing the Kalman gain , reduce the response to abnormal click data and avoid overfitting noise; Kalman gain The expression is as follows:

[0020]

[0021] In the above expression (3), is the time step The prior covariance matrix when , is the learnable parameter observation matrix, is the observation noise covariance matrix of the learnable parameters.

[0022] The original features at time step t include user features, item attributes and context information, which are divided into discrete features and continuous features; the discrete features at time step t are respectively and continuous features Processing; for the i-th discrete feature , using the embedding function Mapping is performed to obtain the embedding vector of discrete features The expression is as follows:

[0023]

[0024] For continuous features , through linear transformation dimensionality reduction, we get the embedding vector of continuous features The expression is as follows:

[0025]

[0026] In the above expression (5), is the weight matrix, is the bias vector.

[0027] Embed discrete features and continuous feature embedding Concatenate to get the integrated feature embedding vector The expression is as follows:

[0028]

[0029] Further dimensionality reduction processing is performed through the fully connected layer to obtain feature data The expression formula is as follows:

[0030]

[0031] In the above expression (7), represents the dimension reduction weight matrix, represents the bias vector, represents the activation function, Represents the feature data at time step t.

[0032] The Kalman gain As a dynamic weighting coefficient and (Right now , which represents the gap between observation and prediction) is multiplied to estimate the prior state Update; get the posterior state estimate at time step t ; Posterior state estimation It is the output of the Kalman filter update phase, representing the prior state estimation after obtaining the feature data at the current moment. The final posterior state estimate at time step t is as follows:

[0033]

[0034] Based on Kalman gain The prior covariance matrix Update to get the posterior covariance matrix at time step t , ends a cycle and Return to step 1 and continue iterating to ensure that the covariance matrix gradually converges with the iteration. Its expression is as follows:

[0035]

[0036] in is the identity matrix.

[0037] As described in step S3, a new loss function is designed by combining Kalman loss and CTR prediction loss:

[0038] In the time steps, using Kalman filtering to obtain , posterior state estimation under simulated time drift, according to the posterior state estimation , which is calculated as follows:

[0039]

[0040] In the above expression (10), yes The scalar representation of express and The cosine distance between them.

[0041] The feature data and the posterior state estimate Input them into the CTR model together, perform forward propagation calculation, and obtain the predicted value of the CTR model Calculate the CTR prediction loss based on the predicted value of the CTR model and the actual situation of user clicks. The cross entropy loss function is used as a representative implementation, and its calculation formula is as follows:

[0042]

[0043] In the above expression (11), is the predicted value of the CTR model; is the target variable, indicating whether the user actually clicks on the item.

[0044] Finally, combine and Calculate the total joint loss function The calculation formula is as follows:

[0045]

[0046] In the above expression (12), It's balance and hyperparameters.

[0047] In step S4, the existing CTR model is trained based on the posterior state estimation and the new loss function to obtain a new CTR model that processes time drift.

[0048] The joint loss function The gradients of the parameters in the CTR model and Kalman filter are calculated by backpropagation. Then, an optimization algorithm is used to update all trainable parameters to minimize the joint loss function. During the update process, the parameters of the CTR model will be adjusted according to the CTR prediction loss. And the joint loss function The back propagation gradient of the Kalman loss is used to adjust the parameters, while the parameters of the Kalman filter module are mainly optimized according to the back propagation gradient of the Kalman loss and the joint loss, so as to achieve the purpose of joint training.

[0049] The model is trained iteratively until it converges or reaches a stopping condition such as the preset number of training rounds. During the training process, hyperparameters can be adjusted based on the performance on the validation set to further improve the model's performance.

[0050] The method proposed in this paper uses a Kalman filter with learnable parameters to capture temporal evolution trends from historical data and update the prior state estimate to simulate the posterior state estimate after time drift. The updated posterior state estimate facilitates model training and enhances the model's ability to handle time drift, thereby improving the model's accuracy and robustness. This method addresses the temporal drift issue in existing CTR prediction models, enabling more accurate CTR predictions. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a system architecture diagram of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below with reference to the accompanying drawings and embodiments, but the present invention is not limited thereto.

[0053] 1. Example:

[0054] Figure 1 is a diagram of the system architecture of the present invention, which consists of two parts: a data augmentation module and a model training and optimization module. In the data augmentation module, a priori state estimates and a priori covariance matrix are calculated based on historical states. The Kalman gain is then calculated to minimize the posterior covariance by balancing the contributions of the predicted state and the observed values. This generates a posterior state estimate after time drift, obtains the posterior covariance matrix of the current state, and continues iteration. In the model training and optimization module, a new loss function is designed by combining the Kalman loss and the CTR prediction loss. The existing CTR model is trained based on the posterior state estimate and this new loss function, resulting in a new CTR model that handles time drift.

[0055] Specifically, in order to illustrate the implementation of the present invention, in the examples, two different datasets, ML-10M and Douban, are selected and used as data objects for subsequent training and optimization of the method combined with the model proposed in this article after preprocessing operations.

[0056] The data augmentation module calculates a priori state estimates and a priori covariance matrix based on historical states. It then calculates the Kalman gain to minimize the posterior covariance by balancing the contributions of the predicted state and the observed values, generating a time-shifted posterior state estimate.

[0057] First, initialize the user interest state vector through the existing CTR model or the user's historical behavior data. Based on the observation of the initial state of the system or prior knowledge, a reasonable estimate is given as the initial value of the state estimation. Based on the historical state, the prior state estimate at the current time step t is predicted, and the current state time step is derived from the posterior state estimate at the previous moment. The prior state estimate , by the time step The posterior state estimate of and the state transfer matrix Multiplying them together, the calculation formula is:

[0058]

[0059] In the above expression (1), is the time step The posterior state estimate of is the state transition matrix at time step t.

[0060] Set the non-zero covariance matrix of the initial state Indicates the uncertainty of the initial state estimate. In general, it can be assumed that the uncertainty of the initial estimate is high and a larger value is chosen. is the initial covariance matrix. By combining with the state transfer matrix and the process noise covariance matrix The calculation of , realizes the iterative update of the predicted state covariance matrix. At time step t, the transfer matrix The posterior covariance matrix for the previous time step t−1 is Processing is performed and the process noise covariance matrix is ​​added , get the prior covariance matrix at time step t predicted by the posterior covariance matrix at time step t-1 , the expression is as follows:

[0061]

[0062] In the above expression (2), the state transfer matrix and are all learnable parameters. Gaussian noise matrix can be applied to simulate random interference in real environment and enhance the robustness of the model.

[0063] The core of Kalman filter-based temporal representation enhancement is to calculate the Kalman gain and simulate the posterior state estimate after time drift to help train the CTR prediction model, thereby enhancing its robustness to time drift and improving the performance of test data in future time steps. The posterior state estimate obtained through Kalman filter and observation value updates helps the CTR prediction model adapt to this drift, thereby improving its performance.

[0064] Calculate Kalman gain , which minimizes the posterior covariance matrix by balancing the contribution of the predicted state and the observed value. In CTR prediction, if the user interest changes and the prediction error expands, the prior covariance matrix Increases, then the Kalman gain As it increases, the model needs to rely more on observation data to correct the prediction. If the advertising platform has an abnormal situation, the observation noise covariance matrix Increase by reducing the Kalman gain , reduce the response to abnormal click data and avoid overfitting noise. Kalman gain The expression is as follows:

[0065]

[0066] In the above expression (3), is the time step The prior covariance matrix when , is the learnable parameter observation matrix, is the observation noise covariance matrix of the learnable parameters.

[0067] The original features at time step t include user features, item attributes and context information, which are divided into discrete features and continuous features. and continuous features For the i-th discrete feature , using the embedding function Mapping is performed to obtain the embedding vector of discrete features The expression is as follows:

[0068]

[0069] For continuous features , through linear transformation dimensionality reduction, we get the embedding vector of continuous features The expression is as follows:

[0070]

[0071] In the above expression (5), is the weight matrix, is the bias vector.

[0072] Embed discrete features and continuous feature embedding Concatenate to get the integrated feature embedding vector The expression is as follows:

[0073]

[0074] Further dimensionality reduction processing is performed through the fully connected layer to obtain feature data The expression formula is as follows:

[0075]

[0076] In the above expression (7), represents the dimension reduction weight matrix, represents the bias vector, represents the activation function, Represents the feature data at time step t.

[0077] The Kalman gain As the dynamic weighting coefficient and the observation residual (i.e. , which represents the gap between observation and prediction) is multiplied to estimate the prior state Update; get the posterior state estimate at time t . Posterior state estimate It is the output of the Kalman filter update phase, representing the prior state estimation after obtaining the feature data at the current moment. The final posterior state estimate at time step t is as follows:

[0078]

[0079] Through the prior covariance matrix With Kalman gain The related calculation of , get the posterior covariance matrix at time step t , ends a cycle and Return to step 1 to calculate the prior covariance matrix for the next time step and continue iterating to ensure that the covariance matrix gradually converges with the iteration. Its expression is as follows:

[0080]

[0081] in is the identity matrix.

[0082] The training and optimization module of the model combines Kalman loss and CTR prediction loss to design a new loss function. Based on the posterior state estimation and the new loss function, the existing CTR model is trained to obtain a new CTR model that processes time drift. time steps, using Kalman filtering to obtain , posterior state estimation under simulated time drift, according to the posterior state estimation and feature data Calculating Kalman loss , which is calculated as follows:

[0083]

[0084] In the above expression (10), yes The scalar representation of express and The cosine distance between them.

[0085] The feature data and the posterior state estimate Input them into the CTR model together, perform forward propagation calculation, and obtain the predicted value of the CTR model Calculate the CTR prediction loss based on the predicted value of the CTR model and the actual situation of user clicks. The cross entropy loss function is used as a representative implementation, and its calculation formula is as follows:

[0086]

[0087] In the above expression (11), is the predicted value of the CTR model; is the target variable, indicating whether the user actually clicks on the item.

[0088] Finally, combine and Calculate the total joint loss function The calculation formula is as follows:

[0089]

[0090] In the above expression (12), It's balance and hyperparameters.

[0091] The joint loss function The gradients of the parameters in the CTR model and Kalman filter are calculated by backpropagation. Then, an optimization algorithm is used to update all trainable parameters to minimize the joint loss function. During the update process, the parameters of the CTR model will be adjusted according to the CTR prediction loss. And the joint loss function The back propagation gradient of the Kalman loss is used to adjust the parameters, while the parameters of the Kalman filter module are mainly optimized according to the back propagation gradient of the Kalman loss and the joint loss, so as to achieve the purpose of joint training.

[0092] The model is trained iteratively until it converges or reaches a stopping condition such as the preset number of training rounds. During the training process, hyperparameters can be adjusted based on the performance on the validation set to further improve the model's performance.

[0093] 2. Experimental Structure and Analysis

[0094] 2.1 Dataset

[0095] 2.1.1ML-10M

[0096] The ML-10M dataset is derived from "Movie Datasets: History and Context" by FM Harper and JA Konstan in the ACM Transactions on Interactive Intelligent Systems. This application method uses ML-10M data from 2002 to 2008 for the new prediction model. The data is divided into six-month time steps based on timestamp information. The first five time steps are used as the training set, the sixth time step as the validation set, and the seventh time step as the test set.

[0097] 2.1.2 Douban

[0098] This application method uses Douban data from 2012 to 2019 for the new prediction model. The data processing method is the same as ML-10M.

[0099] 2.2 Baseline Model

[0100] In order to fully verify the effectiveness of the proposed method for the new prediction model, three benchmark models, ERM, GroupDRO (Distributionally robust neural networks for group shifts), and IRM (Invariant risk minimization), were selected in this section for comparative experiments.

[0101] 1) ERM: Minimize the empirical risk on the training set to obtain the optimal model.

[0102] 2) GroupDRO: Improves out-of-distribution generalization by focusing on the worst-performing group during training, ensuring that the model performs well across all subgroups of the data.

[0103] 3) IRM: Finding invariant predictors in different environments by minimizing the worst-case risk. It encourages the model to learn consistent features across various distributions, improving its robustness to out-of-distribution scenarios.

[0104] 2.3 Experimental Setup

[0105] To make a fair comparison with other baselines, our model is compared with the best results of other baseline models. The learning rate is selected from {0.01, 0.005, 0.001} , choose hyperparameters from {0.1, 0.3, 0.5, 0.7, 0.9} , and set the dimension of the embedding vector to 12. In addition, the labels of ML-10M and Douban represent levels from 1 to 5, and the labels of samples with scores lower than 3 are assigned to 0, and the labels of samples with scores higher than 3 are assigned to 1. The method proposed in this application is not used independently, but should be used in conjunction with existing CTR prediction models. This method is compatible with most CTR prediction models that use feature interactions. In this application, the classic models DeepFM, DCN, and PNN are selected as backbone models to demonstrate the effectiveness of KFTRA.

[0106] 2.4 Results Analysis

[0107] The model of this application conducted comparative experiments on three backbone models on two public datasets, ML-10M and Douban. Table 1 shows the test results of the performance comparison of each model. The overall performance of this method on the two datasets is better than the comparison model. On the ML-10M dataset, the AUC of this model is improved to above 0.748, and the improvement ratio of AUC relative to ERM is good. Among them, DCN has the highest AUC, reaching 0.7491. On PNN, the improvement ratio of AUC relative to ERM is the highest, reaching 0.4%; on the Douban dataset, the AUC of this model is improved to above 0.812. Among them, on the DeepFM and PNN backbone models, the improvement ratio of AUC relative to ERM exceeds 0.55%, and the AUC of PNN reaches 0.8143. And the logloss on both datasets remains at a relatively good level. All indicators have been improved to a certain extent.

[0108] In summary, using the Kalman filter to learn temporal evolution trends and updating the posterior state estimate at the most recent time step to assist model training can effectively improve model performance. Furthermore, while GroupDRO and IRM improve the model's robustness to temporal drift by learning invariant features across time steps, they ignore the temporal evolution of time steps. The Kalman filter effectively captures the temporal trends in state estimates, resulting in significantly better performance than GroupDRO and IRM.

[0109] surface Test results

[0110]

[0111] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. A method for processing CTR prediction time drift based on Kalman filtering, characterized by: The steps include: S1: Calculate the prior state estimate and prior covariance matrix based on the historical state; S2: Calculate the Kalman gain, and then calculate the posterior state estimate and the posterior covariance matrix; S3: Combine Kalman loss and CTR prediction loss to design a new loss function; S4: Train the existing CTR model based on the posterior state estimation and the new loss function to obtain a new CTR model that handles time drift.

2. The method for processing CTR prediction time drift based on Kalman filtering according to claim 1, characterized in that: The calculation of the prior state estimate and the prior covariance matrix based on the historical state in step S1 is: First, initialize the user interest state vector through the existing CTR model or user historical behavior data; give a reasonable estimate based on the observation of the initial state of the system or prior knowledge as the initial value of the state estimation ; Estimate the prior state at the current time step t based on the historical state The calculation formula is: ； in, is the time step The posterior state estimate of is the state transition matrix at time step t; Set the non-zero covariance matrix of the initial state to represent the uncertainty of the initial state estimate; by comparing it with the state transfer matrix and the process noise covariance matrix , to achieve iterative update of the prior covariance matrix; at time step t, the transfer matrix The posterior covariance matrix for the previous time step t−1 is Processing is performed and the process noise covariance matrix is ​​added , get the prior covariance matrix at time step t predicted by the posterior covariance matrix at time step t-1 , the expression is as follows: ； in, and is a learnable parameter.

3. The method for processing CTR prediction time drift based on Kalman filtering according to claim 1, characterized in that: The calculation of the Kalman gain described in step S2, followed by the calculation of the posterior state estimate and the posterior covariance matrix, is: Kalman gain Minimize the posterior covariance matrix by balancing the contribution of predicted state and observed value; in CTR prediction, if the change of user interest leads to the expansion of prediction error, the prior covariance matrix Increases, then the Kalman gain As it increases, the model needs to rely more on observed data to correct the prediction; If an abnormal situation occurs on the advertising platform, the observation noise covariance matrix Increase by reducing the Kalman gain , reduce the response to abnormal click data and avoid overfitting noise; Kalman gain at time step t The calculation formula is shown in (3): ； in, is the learnable parameter observation matrix; Calculate the posterior state estimate and the posterior covariance matrix, which is: Compute the posterior state estimate at time step t , is the output of the Kalman filter update phase, indicating that the feature data at the current moment is obtained After that, the prior state estimate The correction is calculated as follows: ； Indicates that the Kalman gain As a dynamic weighting coefficient multiplied by the observation residual, Indicates the gap between observation and prediction; Get the posterior covariance matrix at time step t , ends a cycle and Return to step S1 and continue iterating to ensure that the covariance matrix gradually converges with the iteration; Its expression is as follows: ； in is the identity matrix.

4. The method for processing CTR prediction time drift based on Kalman filtering according to claim 1, characterized in that: In step S3, a new loss function is designed by combining Kalman loss and CTR prediction loss: First, based on the posterior state estimation obtained by combining Kalman filtering ; The calculation formula is as follows: ； in, yes The scalar representation of express and The cosine distance between The characteristic data at time t and the posterior state estimate Input them into the CTR model together, perform forward propagation calculation, and obtain the predicted value of the CTR model ; Calculate the CTR prediction loss based on the predicted value of the CTR model and the actual situation of user clicks, CTR prediction loss Use the cross entropy loss function as a representative implementation; The calculation formula is as follows: ； in is the predicted value of the CTR model; is the target variable, indicating whether the user actually clicks on the item; Finally, combine and Calculate the total joint loss function , which is calculated as follows: ； in, It's balance and hyperparameters.

5. The method for processing CTR prediction time drift based on Kalman filtering according to claim 1, characterized in that: In step S4, the existing CTR model is trained based on the posterior state estimation and the new loss function to obtain a new CTR model that processes time drift; The joint loss function The gradients of the parameters in the CTR model and Kalman filter are calculated by backpropagation; then, an optimization algorithm is used to update all trainable parameters to minimize the joint loss function. ; During the update process, the parameters of the CTR model will be adjusted according to the CTR prediction loss. And the joint loss function The parameters of the Kalman filter module are mainly optimized according to the back propagation gradient of the Kalman loss and the joint loss, so as to achieve the purpose of joint training; Continuously iterate the training model until the model converges or reaches the preset number of training rounds or other stopping conditions; During the training process, hyperparameters can be adjusted based on the performance on the validation set to further improve the performance of the model.

Images