Double-layer optimization method based on meta-learning in click rate estimation

Abstract

The invention discloses a double-layer optimization method based on meta-learning in click rate estimation, and the method mainly comprises the steps: data selection and vector representation of data, vector representation prediction, calculation of an internal circulation loss value, optimization of a model based on the internal circulation loss value, calculation of an external circulation loss value, and optimization of the model based on the external circulation loss value. And repeating the steps, and finally completing model optimization. According to the invention, by predicting vector representation after future time sequence distribution drifting and utilizing a double-layer optimization mechanism, the model can better adapt to the change of data distribution, and the ability of the click rate estimation model to cope with time sequence distribution drifting is improved, so that the click rate estimation performance in future time steps is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of big data processing, and specifically relates to a double-layer optimization method based on meta-learning in click rate prediction. Background Art

[0002] Click-through rate prediction is one of the important tasks in information retrieval. It estimates the probability of a user clicking on an ad or product after seeing it. This probability value can be used in ranking and recommendation systems to improve user experience and business indicators. At present, research on click-through rate prediction models based on empirical risk minimization has achieved remarkable results, such as DeepFM, DCN, and DIN. These studies usually assume that training and test data are independent and identically distributed, and optimize the click-through rate prediction model by minimizing the prediction error in the training data.

[0003] However, in the real world, this assumption does not hold true because there is a difference between the data in the future time steps and the data in the historical time steps. This difference may be because user interests change over time, causing the distribution of interaction data to change accordingly.

[0004] There are many reasons why the distribution of future data may differ from the distribution of historical data. First, there are some strong correlations in historical data that cannot be generalized to the future. Second, the properties of streaming data often change over time, causing user interaction patterns to dynamically change between historical and future environments, such as the distribution of item popularity. Furthermore, the underlying data distribution of personal interests and background information will gradually change over time, for example, when users repeatedly view the same item over time, their click interest may decrease. These factors lead to differences between historical training data and real-world test data over time, a phenomenon known as time series distribution drift.

[0005] Time series distribution drift negatively affects the performance of the CTR prediction model, which decreases as the time step increases. This shows that models that are well trained on historical data and based on empirical risk minimization are limited by time series distribution drift when generalizing to real-world test data in future time steps. Furthermore, these models based on empirical risk minimization and trained with historical data will cause their performance to deteriorate in future time steps when affected by time series distribution drift. Summary of the invention

[0006] The present invention aims to solve one of the technical problems existing in the related art at least to a certain extent.

[0007] The purpose of the present invention is to provide a two-layer optimization method based on meta-learning in click-through rate prediction, which is suitable for the optimization of most click-through rate prediction models and helps the click-through rate prediction models cope with time series distribution drift.

[0008] In order to achieve the above-mentioned object, the present invention provides a two-layer optimization method based on meta-learning in click rate prediction, comprising: S1. For a click rate prediction model, select a batch of data from the 1st to tth time steps, and use the embedding model to calculate the vector representation of each batch of data; S2, using a feature learner to capture the evolution trend of the vector representation, and predicting the vector representation of the data at the tth time step at the next time step based on the captured evolution trend; S3. Calculate the loss value of the click rate prediction model on these vector representations according to the predicted vector representations of the next time step, and record it as the inner loop loss value; S4. Based on the obtained inner loop loss value, the classifier in the click rate prediction model is optimized using the gradient descent method; S5. Calculate the loss value of the click rate prediction model optimized in step S4 on the true vector representation at the next time step, and record it as the outer loop loss value. S6. Based on the obtained outer loop loss value, the gradient descent method is used again to optimize the classifier in the click rate prediction model; S7. Repeat steps S1 to S6 until the model converges, thereby completing the optimization of the click rate prediction model.

[0009] A further preferred technical solution of the present invention is that the click rate prediction model in step S1 is composed of an embedding model and 1 classifier Composition, among which, represents the input of the model, and They represent the parameters of the embedding model and the classifier respectively; for a given user-item interaction feature, the embedding model first maps the interaction feature into a vector representation, and then the classifier maps the vector representation into the user's click probability for the item.

[0010] Preferably, the embedding model in step S1 converts the data of the tth time step into Mapping is represented by a vector, and the specific calculation formula is: ; in, Representation Embedding Model Parameters, represents the original data at the tth time step, represents the vector representation of the data at the t-th time step, is the mapping function.

[0011] Preferably, the feature learner in step S2 is a model built based on the attention mechanism; the feature learner is represented by a vector of historical moments As input, the vector at the next moment is represented by As output, the output is the feature learner's prediction of the future vector representation; The feature learner predicts the vector representation of the data at the tth time step in the next time step based on the captured evolution trend. The calculation formula is: ; ; ; In the formula, , and are all learnable parameters. represents the query matrix, which is used to extract query information from the input sequence. represents the key matrix, which is used to extract key information from the input sequence. Represents a value matrix, which is used to extract value information from the input sequence; is a hyperparameter, indicating the number of attention heads, with a value range of ; and are all learnable parameters in the linear layer, represents the activation function of the linear layer; Indicates i The feature representation and j The attention weights between feature representations, Indicates i The feature representation and k Attention weights between feature representations; , Represent the attention weight and the output result of the attention mechanism calculation respectively; It is the vector representation of the predicted next moment.

[0012] Preferably, the calculation formula of the inner loop loss value in step S3 is: ; In the formula, is a hyperparameter, which represents the weight of the loss function and takes the value , represents the regularization weight; Indicates that when the input data is , and the vector representation of the data is When , the parameter is The output of the classifier; Represents the output results and labels of the calculation model The two-way cross entropy loss value of ; Represents regularization, which is used to make the classifier The predicted output on The predicted output on is close to; is a scalar, The calculation formula is: ; in Indicates that when the input data is , and the vector representation of the data is When , the parameter is The output of the classifier; is a hyperparameter, and its value range is 1 to 5. Preferably, in step S4, based on the obtained inner loop loss value, when the classifier in the click-through rate prediction model is optimized using the gradient descent method, the classifier parameter update is expressed as: ; in, Indicates the inner loop loss value For classifier parameters The gradient of It is a hyperparameter, indicating the inner loop learning rate, and its value range is 0.0001~0.1; Represents the parameters of the classifier after optimization based on the inner loop loss value.

[0013] Preferably, the calculation formula of the external circulation loss value in step S5 is: ; in, Indicates that when the input data is , and the vector representation of the data is When , the parameter is The output of the classifier; Indicates the output and label of the calculation classifier The two-way cross entropy loss value.

[0014] Preferably, in step S6, when the classifier in the click rate prediction model is optimized using the gradient descent method based on the obtained outer loop loss value, the classifier parameter update is expressed as: ; in, Represents the gradient of the outer loop loss value to all parameters in the click rate prediction model; It is a hyperparameter, indicating the outer loop learning rate, and its value range is 0.0001~0.1; represents all model parameters that have not been updated in step S4, Represents all model parameters updated using the outer loop loss value.

[0015] Another aspect of the present invention provides a non-transitory computer-readable storage medium having computer instructions stored thereon, which enable a computer to execute the above-mentioned two-layer optimization method based on meta-learning in click-through rate estimation.

[0016] Yet another aspect of the present invention provides an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus, and the processor calls logic instructions in the memory to execute the above-mentioned two-layer optimization method based on meta-learning in click-through rate estimation.

[0017] On the other hand, the present invention provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer executes the above-mentioned two-layer optimization method based on meta-learning in click-through rate estimation.

[0018] Beneficial effect: The present invention predicts the vector representation after the future time series distribution drift and utilizes a two-layer optimization mechanism to enable the model to better adapt to changes in data distribution, thereby improving the ability of the click-through rate prediction model to cope with the time series distribution drift, thereby improving the click-through rate prediction performance at future time steps.

[0019] The optimization method of the present invention not only focuses on the performance of the model on historical data, but also pays more attention to the generalization ability of the model on future real data, ensuring that the model can achieve good results in practical applications and improving the generalization ability of the model. Therefore, the optimization method of the present invention is applicable to most click-through rate prediction models, such as DeepFM, DCN and DIN, and has good versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flowchart for implementing the two-layer optimization method based on meta-learning in the click rate prediction of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0022] Combine the following Figure 1 The present invention describes a two-layer optimization method based on meta-learning in click rate prediction.

[0023] Embodiment 1: This embodiment provides a two-layer optimization method based on meta-learning in click rate estimation. The main steps of the method are: Data selection and vector representation of data, prediction of vector representation, calculation of inner loop loss value, optimization model based on inner loop loss value, calculation of outer loop loss value, optimization model based on outer loop loss value, and repeating the above steps to finally complete model optimization.

[0024] like Figure 1 As shown, each step is described in detail below: S1. For a click rate prediction model, select a batch of data from the 1st to tth time steps, and use the embedding model to calculate the vector representation of each batch of data.

[0025] In this embodiment, the click rate prediction model consists of an embedding model and 1 classifier Composition, among which, represents the input of the model, and Represent the parameters of the embedding model and the classifier respectively; for a given user and item interaction feature, the embedding model first maps the interaction feature to a vector representation, and then the classifier maps the vector representation to the user's click probability on the item. The embedding model converts the data at the tth time step into Mapping is represented by a vector, and the specific calculation formula is: ; in, Representation Embedding Model Parameters, represents the original data at the tth time step, represents the vector representation of the data at the t-th time step, Represents a mapping function.

[0026] S2. Using a feature learner to capture the evolution trend of the vector representation, and predicting the vector representation of the data at the tth time step at the next time step based on the captured evolution trend.

[0027] The above feature learner is a model built based on the attention mechanism. The feature learner is represented by a vector of historical moments As input, the vector at the next moment is represented by As output, the output is the feature learner's prediction of the future vector representation; The feature learner predicts the vector representation of the data at the tth time step in the next time step based on the captured evolution trend. The calculation formula is: ; ; ; In the formula, , and are all learnable parameters. represents the query matrix, which is used to extract query information from the input sequence. represents the key matrix, which is used to extract key information from the input sequence. Represents a value matrix, which is used to extract value information from the input sequence; is a hyperparameter, indicating the number of attention heads, with a value range of ; and are all learnable parameters in the linear layer, represents the activation function of the linear layer; Indicates the s i The feature representation and j The attention weights between feature representations, Indicates i The feature representation and k Attention weights between feature representations; , Represent the attention weight and the output result of the attention mechanism calculation respectively; It is the vector representation of the predicted next moment.

[0028] S3. According to the predicted vector representation of the next time step, the loss value of the click rate prediction model on these vector representations is calculated, which is recorded as the inner loop loss value.

[0029] The calculation formula of the inner loop loss value is: ; In the formula, is a hyperparameter, which represents the weight of the loss function and takes the value , represents the regularization weight; Indicates that when the input data is , and the vector representation of the data is When , the parameter is The output of the classifier; Represents the output results and labels of the calculation model The two-way cross entropy loss value of ; Represents regularization, which is used to make the classifier The predicted output on The predicted output on is close to; is a scalar, The calculation formula is: ; in Indicates that when the input data is , and the vector representation of the data is When , the parameter is The output of the classifier; It is a hyperparameter, and its value range is 1~5.

[0030] S4. Based on the obtained inner loop loss value, the gradient descent method is used to optimize the classifier in the click rate prediction model.

[0031] The classifier parameter update is expressed as: ; in, Indicates the inner loop loss value For classifier parameters The gradient of It is a hyperparameter, indicating the inner loop learning rate, and its value range is 0.0001~0.1; Represents the parameters of the classifier after optimization based on the inner loop loss value.

[0032] S5. Based on the CTR prediction model optimized in step S4, calculate its loss value on the true vector representation at the next time step, which is recorded as the outer loop loss value.

[0033] The calculation formula of the external circulation loss value is: ; in, Indicates that when the input data is , and the vector representation of the data is When , the parameter is The output of the classifier; Indicates the output and label of the calculation classifier The two-way cross entropy loss value.

[0034] S6. Based on the obtained outer loop loss value, the gradient descent method is used again to optimize the classifier in the click rate prediction model.

[0035] The classifier parameter update is expressed as: ; in, Represents the gradient of the outer loop loss value to all parameters in the click rate prediction model; It is a hyperparameter, indicating the outer loop learning rate, and its value range is 0.0001~0.1; represents all model parameters that have not been updated in step S4, Represents all model parameters updated using the outer loop loss value.

[0036] S7. Repeat steps S1 to S6 until the model converges, thereby completing the optimization of the click rate prediction model.

[0037] The above optimization method can improve the model's ability to cope with time series distribution drift. In this embodiment, Douba and ML-10M public data sets are selected as research objects, DeepFM, DCN, PNN and AFM are selected as basic models, and AUC and Logloss are used as evaluation indicators, where the larger the AUC, the better the model effect, and the smaller the Logloss, the better the model effect. ERM represents the use of traditional deep learning optimization methods, and TDAF represents the use of a two-layer optimization method based on meta-learning in the click-through rate estimation involved in the present invention. Imp represents the percentage increase in AUC when the method of the present invention is used compared to the traditional optimization method. The experimental results shown in Table 1 show that the present invention can effectively improve the performance of the basic model.

[0038] Table 1 Verification results of Douba and ML-10M public datasets

[0039] Embodiment 2: This embodiment provides a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions causing a computer to execute a two-layer optimization method based on meta-learning in click-through rate estimation, the method comprising the following steps: S1. For a click rate prediction model, select a batch of data from the 1st to tth time steps, and use the embedding model to calculate the vector representation of each batch of data; S2, using a feature learner to capture the evolution trend of the vector representation, and predicting the vector representation of the data at the tth time step at the next time step based on the captured evolution trend; S3. Calculate the loss value of the click rate prediction model on these vector representations according to the predicted vector representations of the next time step, and record it as the inner loop loss value; S4. Based on the obtained inner loop loss value, the classifier in the click rate prediction model is optimized using the gradient descent method; S5. Calculate the loss value of the click rate prediction model optimized in step S4 on the true vector representation at the next time step, and record it as the outer loop loss value. S6. Based on the obtained outer loop loss value, the gradient descent method is used again to optimize the classifier in the click rate prediction model; S7. Repeat steps S1 to S6 until the model converges, thereby completing the optimization of the click rate prediction model.

[0040] Embodiment 3: This embodiment provides an electronic device, which may include: a processor, a communications interface, a memory, and a communication bus, wherein the processor, the communications interface, and the memory communicate with each other via the communication bus. The processor may call the logic instructions in the memory to execute a two-layer optimization method based on meta-learning in click-through rate estimation, and the method includes the following steps: S1. For a click rate prediction model, select a batch of data from the 1st to tth time steps, and use the embedding model to calculate the vector representation of each batch of data; S2, using a feature learner to capture the evolution trend of the vector representation, and predicting the vector representation of the data at the tth time step at the next time step based on the captured evolution trend; S3. Calculate the loss value of the click rate prediction model on these vector representations according to the predicted vector representations of the next time step, and record it as the inner loop loss value; S4. Based on the obtained inner loop loss value, the classifier in the click rate prediction model is optimized using the gradient descent method; S5. Calculate the loss value of the click rate prediction model optimized in step S4 on the true vector representation at the next time step, and record it as the outer loop loss value. S6. Based on the obtained outer loop loss value, the gradient descent method is used again to optimize the classifier in the click rate prediction model; S7. Repeat steps S1 to S6 until the model converges, thereby completing the optimization of the click rate prediction model.

[0041] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0042] Embodiment 4: This embodiment provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a two-layer optimization method based on meta-learning in click-through rate estimation. The method includes the following steps: S1. For a click rate prediction model, select a batch of data from the 1st to tth time steps, and use the embedding model to calculate the vector representation of each batch of data; S2, using a feature learner to capture the evolution trend of the vector representation, and predicting the vector representation of the data at the tth time step at the next time step based on the captured evolution trend; S3. Calculate the loss value of the click rate prediction model on these vector representations according to the predicted vector representations of the next time step, and record it as the inner loop loss value; S4. Based on the obtained inner loop loss value, the classifier in the click rate prediction model is optimized using the gradient descent method; S5. Calculate the loss value of the click rate prediction model optimized in step S4 on the true vector representation at the next time step, and record it as the outer loop loss value. S6. Based on the obtained outer loop loss value, the gradient descent method is used again to optimize the classifier in the click rate prediction model; S7. Repeat steps S1 to S6 until the model converges, thereby completing the optimization of the click rate prediction model.

[0043] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0044] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A two-layer optimization method based on meta-learning in click-through rate prediction, characterized in that: include: S1. For a click rate prediction model, select a batch of data from the 1st to tth time steps, and use the embedding model to calculate the vector representation of each batch of data; S2, using a feature learner to capture the evolution trend of the vector representation, and predicting the vector representation of the data at the tth time step at the next time step based on the captured evolution trend; S3. Calculate the loss value of the click rate prediction model on these vector representations according to the predicted vector representations of the next time step, and record it as the inner loop loss value; S4. Based on the obtained inner loop loss value, the classifier in the click rate prediction model is optimized using the gradient descent method; S5. Calculate the loss value of the click rate prediction model optimized in step S4 on the true vector representation at the next time step, and record it as the outer loop loss value. S6. Based on the obtained outer loop loss value, the gradient descent method is used again to optimize the classifier in the click rate prediction model; S7. Repeat steps S1 to S6 until the model converges, thereby completing the optimization of the click rate prediction model.

2. The two-layer optimization method based on meta-learning in click-through rate estimation according to claim 1, characterized in that: The click rate prediction model described in step S1 consists of an embedding model and 1 classifier Composition, among which, represents the input of the model, and They represent the parameters of the embedding model and the classifier respectively; for a given user-item interaction feature, the embedding model first maps the interaction feature into a vector representation, and then the classifier maps the vector representation into the user's click probability for the item.

3. The two-layer optimization method based on meta-learning in click-through rate prediction according to claim 2, characterized in that: In step S1, the embedding model converts the data of the tth time step into Mapping is represented by a vector, and the specific calculation formula is: ； in, Representation Embedding Model Parameters, represents the original data at the tth time step, represents the vector representation of the data at the t-th time step, is the mapping function.

4. The two-layer optimization method based on meta-learning in click-through rate prediction according to claim 1, characterized in that: The feature learner in step S2 is a model built based on the attention mechanism; the feature learner is represented by a vector of historical moments As input, the vector at the next moment is represented by As output, the output is the feature learner's prediction of the future vector representation; The feature learner predicts the vector representation of the data at the tth time step in the next time step based on the captured evolution trend. The calculation formula is: ； ； ； In the formula, , and are all learnable parameters. represents the query matrix, which is used to extract query information from the input sequence. represents the key matrix, which is used to extract key information from the input sequence. Represents a value matrix, which is used to extract value information from the input sequence; is a hyperparameter, indicating the number of attention heads, with a value range of ; and are all learnable parameters in the linear layer, represents the activation function of the linear layer; Indicates i The feature representation and j The attention weights between feature representations, Indicates i The feature representation and k Attention weights between feature representations; , Represent the attention weight and the output result of the attention mechanism calculation respectively; It is the vector representation of the predicted next moment.

5. The two-layer optimization method based on meta-learning in click-through rate prediction according to claim 1, characterized in that: The calculation formula of the inner loop loss value in step S3 is: ; In the formula, is a hyperparameter, which represents the weight of the loss function and takes the value , represents the regularization weight; Indicates that when the input data is , and the vector representation of the data is When , the parameter is The output of the classifier; Represents the output results and labels of the calculation model The two-way cross entropy loss value of ; Represents regularization, which is used to make the classifier The predicted output on The predicted output on is close to; is a scalar, The calculation formula is: ； in Indicates that when the input data is , and the vector representation of the data is When , the parameter is The output of the classifier; It is a hyperparameter, and its value range is 1~5.

6. The two-layer optimization method based on meta-learning in click-through rate prediction according to claim 5, characterized in that: In step S4, based on the obtained inner loop loss value, when the classifier in the click rate prediction model is optimized using the gradient descent method, the classifier parameter update is expressed as: ； in, Indicates the inner loop loss value For classifier parameters The gradient of It is a hyperparameter, indicating the inner loop learning rate, and its value range is 0.0001~0.1; Represents the parameters of the classifier after optimization based on the inner loop loss value.

7. The two-layer optimization method based on meta-learning in click-through rate prediction according to claim 1, characterized in that: The calculation formula of the external circulation loss value in step S5 is: ； in, Indicates that when the input data is , and the vector representation of the data is When , the parameter is The output of the classifier; Indicates the output and label of the calculation classifier The two-way cross entropy loss value of .

8. The two-layer optimization method based on meta-learning in click-through rate prediction according to claim 7, characterized in that: In step S6, based on the obtained outer loop loss value, when the classifier in the click rate prediction model is optimized using the gradient descent method, the classifier parameter update is expressed as: ； in, Represents the gradient of the outer loop loss value to all parameters in the click rate prediction model; It is a hyperparameter, indicating the outer loop learning rate, and its value range is 0.0001~0.1; represents all model parameters that have not been updated in step S4, Represents all model parameters updated using the outer loop loss value.

9. A non-transitory computer-readable storage medium, characterized in that: Computer instructions are stored thereon, which enable the computer to execute the two-layer optimization method based on meta-learning in click rate estimation described in any one of claims 1-8.

10. An electronic device, characterized in that: include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus, and the processor calls the logic instructions in the memory to execute the two-layer optimization method based on meta-learning in the click rate prediction described in any one of claims 1-8.

Images