A click rate prediction method and device based on multi-level feature interaction

Abstract

The present invention discloses a method and device for predicting click rate based on multi-level feature interaction, the method comprising: collecting user feature data and historical behavior information to construct a training data set; adding a second-order feature interaction layer between the feature embedding layer and the fully connected layer of the xDeepFM model to improve the xDeepFM model, and using the improved xDeepFM model as a click rate prediction model; using the training data set to train the click rate prediction model; using the trained click rate prediction model to predict the click rate to be predicted to obtain a prediction result. The present invention can improve the accuracy of click rate prediction and reduce the model training time.

Description

Technical Field

[0001] The present invention relates to the technical fields of recommendation algorithms and click-through rate prediction, and in particular to a click-through rate prediction method and device based on multi-level feature interaction. Background Art

[0002] Click-through rate (CTR) prediction is the most important component in both recommendation systems and advertising systems. In recommendation systems, the goal is to maximize the number of clicks, and the order of items on the interface returned to the user should be ranked according to CTR predictions; in advertising systems, the main goal is to increase advertising revenue, and advertising placement should also be ranked according to advertising CTR predictions. Therefore, correctly predicting CTR is crucial.

[0003] The implicit feature interaction in user click behavior is very important for CTR prediction, and feature engineering plays a decisive role. Traditional machine learning algorithms must make full use of all input features to achieve ideal results. However, it is difficult to achieve the best results by directly using raw data. Excellent data scientists are very accurate in the combination of features and play a key role in model building. However, this process is costly because a large amount of engineering data and rich knowledge in related fields are required to achieve good results. At the same time, these solutions lack generalization capabilities and are difficult to promote in other fields. Therefore, existing CTR prediction models do not fully utilize low-order feature information and have too many training parameters. Summary of the invention

[0004] The present invention provides a click-through rate prediction method and device based on multi-level feature interaction, so as to solve the technical problems that the existing CTR prediction model does not fully utilize low-order feature information and has too many training parameters.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] On the one hand, the present invention provides a click rate prediction method based on multi-level feature interaction, comprising:

[0007] Collect user feature data and historical behavior information to build a training data set;

[0008] A second-order feature interaction layer is added between the feature embedding layer and the fully connected layer of the xDeepFM model to improve the xDeepFM model, and the improved xDeepFM model is used as the click rate prediction model;

[0009] Using the training data set to train the click rate prediction model;

[0010] The trained click-through rate prediction model is used to predict the click-through rate to be predicted and obtain the prediction result.

[0011] Furthermore, the user's characteristic data and historical behavior information are collected to construct a training data set, including:

[0012] Collecting user feature data and historical behavior information; wherein the feature data includes: user age, user gender, mobile phone model used by the user and user location information; the historical behavior information includes: user browsing time and whether the user clicks on advertisements;

[0013] The collected feature data and historical behavior information are preprocessed to extract continuous features and classification features respectively, and the continuous features are placed before the classification features to construct a training data set.

[0014] Furthermore, the click rate prediction model includes a feature embedding layer, a compressed interaction network, a second-order feature interaction layer, a fully connected layer and a prediction layer; wherein,

[0015] The feature embedding layer is used to convert the high-dimensional sparse feature vector into a low-dimensional dense feature vector, and input the converted feature vector into the compression interaction network and the second-order feature interaction layer;

[0016] The second-order feature interaction layer is used to multiply the vectors in the n domains in pairs to obtain n*(n-1) / 2 vectors, and add these vectors to obtain a low-dimensional feature vector y e , calculated as follows:

[0017]

[0018] in, represents the element-wise product of two vectors, e i represents the embedding vector of the ith domain, e j represents the embedding vector of the jth domain, v i represents the weight of the ith domain, v j represents the weight of the jth domain;

[0019] The second-order feature interaction layer inputs the obtained feature vector into the fully connected layer to learn high-order feature interactions. The calculation method is as follows:

[0020]

[0021] Where l is the number of the hidden layer, is a nonlinear activation function, is the output vector of the lth hidden layer, represents the output vector of the l-1th hidden layer, W (l)represents the weight matrix of the lth hidden layer, b (l) Represents the bias of the lth hidden layer;

[0022] The prediction layer integrates linear regression, the compressed interaction network and the fully connected layer into one framework, and the calculation method is as follows:

[0023]

[0024] in, represents the output of the prediction layer, σ is the Sigmoid function, x represents the original feature value that has not been processed by the feature embedding layer, and y dnn ,y cin are the output values ​​of the fully connected layer and the compressed interaction network, respectively, linear , W dnn and W cin are the parameter models of linear regression, the fully connected layer and the compressed interaction network, respectively, and b dnn and b are the bias parameters of the fully connected layer and the prediction layer, respectively.

[0025] Furthermore, the click rate prediction method based on multi-level feature interaction also includes:

[0026] The maximum support dimension parameter is introduced in the one-hot encoding process of the click rate prediction model to prevent dimension explosion. The dictionary mapping relationship between the original data and the occurrence frequency is as follows:

[0027] P[x i ]=N i ,i∈{1,2,...,K}

[0028] Among them, x i is the ith different hash value, N i For x i The corresponding frequency, K is the total number of hash values; if K≤M, then just change x i The corresponding index takes the value of 1 in the one-hot encoding. If K>M, the frequency N needs to be i Perform secondary encoding and create a new mapping table until the dimension is less than M; P[x i ] represents the frequency of occurrence of the i-th feature.

[0029] Furthermore, the click rate prediction method based on multi-level feature interaction also includes:

[0030] In the training process of the click rate prediction model, the early stopping method is used to select the optimal number of iterations; wherein the expression of the early stopping method is as follows:

[0031]

[0032] Among them, E opt (t) and E va (t) are the best validation set error and test set error at iteration t; when the generalization loss GL(t) exceeds a certain threshold, training is stopped.

[0033] On the other hand, the present invention also provides a click rate prediction device based on multi-level feature interaction, comprising:

[0034] The data set construction module is used to collect user feature data and historical behavior information to construct a training data set;

[0035] The xDeepFM model improvement module is used to add a second-order feature interaction layer between the feature embedding layer and the fully connected layer of the xDeepFM model to improve the xDeepFM model, and use the improved xDeepFM model as a click-through rate prediction model;

[0036] A model training module, used to train the click rate prediction model using the training data set;

[0037] The click rate prediction module is used to predict the click rate to be predicted using the trained click rate prediction model to obtain the prediction result.

[0038] Furthermore, the data set construction module is specifically used for:

[0039] Collecting user feature data and historical behavior information; wherein the feature data includes: user age, user gender, mobile phone model used by the user and user location information; the historical behavior information includes: user browsing time and whether the user clicks on advertisements;

[0040] The collected feature data and historical behavior information are preprocessed to extract continuous features and classification features respectively, and the continuous features are placed before the classification features to construct a training data set.

[0041] Furthermore, the click rate prediction model includes a feature embedding layer, a compressed interaction network, a second-order feature interaction layer, a fully connected layer and a prediction layer; wherein,

[0042] The feature embedding layer is used to convert the high-dimensional sparse feature vector into a low-dimensional dense feature vector, and input the converted feature vector into the compression interaction network and the second-order feature interaction layer;

[0043] The second-order feature interaction layer is used to multiply the vectors in the n domains in pairs to obtain n*(n-1) / 2 vectors, and add these vectors to obtain a low-dimensional feature vector y e , calculated as follows:

[0044]

[0045] in, represents the element-wise product of two vectors, e i represents the embedding vector of the ith domain, e j represents the embedding vector of the jth domain, v i represents the weight of the ith domain, v j represents the weight of the jth domain;

[0046] The second-order feature interaction layer inputs the obtained feature vector into the fully connected layer to learn high-order feature interactions. The calculation method is as follows:

[0047]

[0048] Where l is the number of the hidden layer, is a nonlinear activation function, is the output vector of the lth hidden layer, represents the output vector of the l-1th hidden layer, W (l) represents the weight matrix of the lth hidden layer, b (l) Represents the bias of the lth hidden layer;

[0049] The prediction layer integrates linear regression, the compressed interaction network and the fully connected layer into one framework, and the calculation method is as follows:

[0050]

[0051] in, represents the output of the prediction layer, σ is the Sigmoid function, x represents the original feature value that has not been processed by the feature embedding layer, and y dnn ,y cin are the output values ​​of the fully connected layer and the compressed interaction network, respectively, linear , W dnn and W cin are the parameter models of linear regression, the fully connected layer and the compressed interaction network, respectively, and b dnn and b are the bias parameters of the fully connected layer and the prediction layer, respectively.

[0052] Furthermore, the click rate prediction device based on multi-level feature interaction also includes:

[0053] The module for preventing dimension explosion is used to introduce a maximum support dimension parameter in the one-hot encoding process of the click rate prediction model to prevent dimension explosion. The dictionary mapping relationship between the original data and the occurrence frequency is as follows:

[0054] P[x i ]=N i ,i∈{1,2,...,K}

[0055] Among them, x i is the ith different hash value, N i For x i The corresponding frequency, K is the total number of hash values; if K≤M, then just change x i The corresponding index takes the value of 1 in the one-hot encoding. If K>M, the frequency N needs to be i Perform secondary encoding and create a new mapping table until the dimension is less than M; P[x i ] represents the frequency of occurrence of the i-th feature.

[0056] Furthermore, the click rate prediction device based on multi-level feature interaction also includes:

[0057] The optimal number of iterations selection module is used to use the early stopping method to select the optimal number of iterations during the training process of the click rate prediction model; wherein the expression of the early stopping method is as follows:

[0058]

[0059] Among them, E opt (t) and E va (t) are the best validation set error and test set error at iteration t; when the generalization loss GL(t) exceeds a certain threshold, training is stopped.

[0060] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.

[0061] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, and the instruction is loaded and executed by a processor to implement the above method.

[0062] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0063] Compared with the xDeepFM model, the improved algorithm of the present invention makes better use of feature information; fewer parameters need to be trained, there is no dimensionality explosion phenomenon in the training phase, and the risk of overfitting in model training is avoided, thereby improving the accuracy of click-through rate prediction and having better overall performance than existing advanced models. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0065] Figure 1 It is a schematic diagram of the execution flow of the click rate prediction method based on multi-level feature interaction provided by an embodiment of the present invention;

[0066] Figure 2 Schematic diagram of the network structure of the click rate prediction model provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0068] First embodiment

[0069] This embodiment provides a click rate prediction method based on multi-level feature interaction. This embodiment continues the idea of ​​using a fusion structure machine learning model to predict click rates, and proposes to establish a model that integrates multi-level feature interactions to predict click rates. The method can be implemented by an electronic device, which can be a terminal or a server. The execution process of the method is as follows: Figure 1 As shown, the following steps are included:

[0070] S1, collect user feature data and historical behavior information to build a training data set;

[0071] Specifically, in this embodiment, the above S1 includes the following process:

[0072] S11, collecting characteristic data and historical behavior information of users to create a data set; wherein the collected user characteristic data includes: user age, user gender, mobile phone model used by users, and user location information; the collected historical behavior information includes: user browsing time and whether the user clicks on advertisements;

[0073] S12, preprocessing the data set obtained in S11, extracting continuous features and categorical features respectively, and placing the continuous features before the categorical features to construct a training data set.

[0074] It should be noted that the continuous features here refer to features such as age, such as 18, 22, 23, 24...; categorical features can be gender, that is, male, female, and other; these features have been determined to be of different types before processing, so they can be implemented using existing methods, so they will not be described here.

[0075] S2, adding a second-order feature interaction layer between the feature embedding layer and the fully connected layer of the xDeepFM model to improve the xDeepFM model, and using the improved xDeepFM model as the click rate prediction model;

[0076] Specifically, the network structure of the click rate prediction model of this embodiment is as follows: Figure 2 As shown, it includes feature embedding layer, compression interaction network, second-order feature interaction layer, fully connected layer and prediction layer; among them,

[0077] The feature embedding layer converts the high-dimensional sparse feature vector into a low-dimensional dense feature vector, which is represented as follows:

[0078] e=[e1,e2,...,e n ]

[0079] Where n represents the number of domains, e i represents the embedding vector of the i-th domain;

[0080] The feature vector obtained by the feature embedding layer is input into the compression interaction network and the second-order feature interaction layer;

[0081] The input of the compressed interaction network comes from the feature embedding layer, and the specific calculation method is as follows:

[0082]

[0083] Among them, the matrix X 0 ∈R n×D , n, D represent the number of domains and the embedding vector dimension of the domain respectively. The i-th row in the matrix represents the embedding vector representation of the i-th domain: X l represents the state of the lth hidden layer in the compressed interaction network, H l represents the number of compressed features in the lth hidden layer, and the feature embedding layer is called the 0th hidden layer, that is, H0 = n; 1≤h≤H l , is the variable parameter of the hth hidden feature in the lth hidden layer, It is the product operation of corresponding elements between two vectors.

[0084] The second-order feature interaction layer multiplies the vectors in the n domains in pairs to obtain n*(n-1) / 2 vectors, and adds these vectors to obtain a low-dimensional feature vector y e , the specific calculation method is as follows:

[0085]

[0086] in, represents the element-wise product of two vectors, i.e. e i represents the embedding vector of the ith domain, e j represents the embedding vector of the jth domain, v i represents the weight of the ith domain, v j represents the weight of the jth domain;

[0087] The second-order feature interaction layer inputs the obtained feature vector into the fully connected layer to learn high-order feature interactions. The specific calculation method is as follows:

[0088]

[0089] Where l is the number of the hidden layer, is a nonlinear activation function, is the output vector of the lth hidden layer, represents the output vector of the l-1th hidden layer, W (l) represents the weight matrix of the lth hidden layer, b (l) Represents the bias of the lth hidden layer;

[0090] In the prediction layer, linear regression, compressed interaction network and fully connected layer are integrated into one framework. The specific calculation method is as follows:

[0091]

[0092] in, represents the output of the prediction layer, σ is the Sigmoid function, x represents the original feature value that has not been processed by the feature embedding layer, and y dnn ,y cin are the output values ​​of the fully connected layer and the compressed interaction network, W linear , W dnn and W cin are the parameter models of linear regression, fully connected layer and compressed interaction network, b dnn and b are the bias parameters of the fully connected layer and the prediction layer, respectively.

[0093] S3, using the training data set to train the click rate prediction model;

[0094] It should be noted that, in this embodiment, in order to prevent the dimensionality explosion phenomenon in the training stage and avoid the risk of overfitting in model training, this embodiment also introduces a maximum support dimension parameter M in the one-hot encoding process of the click-through rate prediction model to prevent the dimensionality explosion phenomenon; and, in the training process of the click-through rate prediction model, the early stopping method is used to select the optimal number of iterations.

[0095] The maximum supported dimension parameter M is a self-set value, usually between 100-1000. In this embodiment, its value is set to 500.

[0096] The dictionary mapping relationship between the original data and the frequency of occurrence is as follows:

[0097] P[x i ]=N i ,i∈{1,2,...,K}

[0098] Among them, x i is the ith different hash value, N i For x i The corresponding frequency, K is the total number of hash values; if K≤M, then just change x i The corresponding index takes the value of 1 in the one-hot encoding. If K>M, the frequency N needs to be i Perform secondary encoding and create a new mapping table until the dimension is less than M; P[x i ] represents the frequency of occurrence of the i-th feature.

[0099] For example, if the gender characteristics of users in the data set are male, female, and other, which are three categories and are less than the set maximum supported dimension parameter, the simplest encoding method is used, 001 for male, 010 for female, and 100 for others; a three-dimensional matrix, each index represents a category; and if it is a feature of the items that users have browsed, if there are 600 items, the frequency statistics method P[x]=n is used. For example, if a book appears 30 times and a cola appears 20 times, P[book]=30, P[cola]=20, and so on. The frequency of occurrence is sorted and only the first 500 items are taken, and the 100 items with lower frequencies are discarded to achieve the purpose of dimensionality reduction.

[0100] The expression of early stopping method is as follows:

[0101]

[0102] Among them, E opt (t) and E va (t) are the best validation set error and test set error at iteration t; when the generalization loss GL(t) exceeds a certain threshold, training is stopped.

[0103] S4, using the trained click rate prediction model to predict the click rate to be predicted, and obtain the prediction result.

[0104] The following two data sets are used to compare the effects of the model in this embodiment and other models:

[0105] 1) Criteo dataset: The Criteo dataset is a real dataset about the historical click behavior of users on advertisements provided by Criteo. The dataset includes 45 million user click event records, 13 continuous features and 26 categorical features.

[0106] 2) Avazu dataset: The Avazu dataset is a real dataset about the historical click behavior of users on advertisements provided by Avazu. The dataset includes 60 million user click event records, 8 continuous features and 14 categorical features.

[0107] LogLoss and AUC are used to evaluate the performance of the model. The experimental results are shown in Table 1.

[0108] Logloss: It is the predicted value of a single sample. The calculation result is the binary cross entropy between the predicted probability and the true value, which can reflect the gap between the predicted value and the true value. The smaller the better.

[0109] AUC: Randomly select a pair of samples, including one positive sample and one negative sample respectively. The probability that the model ranks the positive sample above the negative sample is the AUC. It can reflect the relative relationship between samples and will not be disturbed by changes in specific probability values. The larger the better.

[0110] Table 1 Comparison results of the model proposed in this example with other advanced models

[0111]

[0112]

[0113] Experimental results show that the click-through rate prediction model based on multi-level feature interaction proposed in this embodiment has better performance than other existing models in terms of Logloss and AUC.

[0114] In summary, based on the xDeepFM model, this embodiment proposes a click rate prediction model using multi-level feature interaction. The model introduces a second-order feature interaction layer between the DNN and the embedding layer, fully integrates low-order and high-order features, and reduces the number of parameters by multiplying the second-order feature vectors, thereby improving the model's feature combination capability; and can simultaneously learn high-order feature vectors explicitly and implicitly, using vectors as the finest granularity to learn feature correlations.

[0115] Compared with the existing click-through rate prediction method based on fusion structure, the click-through rate prediction method based on multi-level feature interaction proposed in this embodiment can utilize richer high-order feature combination information. A large number of experiments based on real scene data sets show that the click-through rate prediction method based on multi-level feature interaction proposed in this embodiment can achieve better prediction performance than the existing click-through rate prediction method based on fusion structure.

[0116] Second embodiment

[0117] This embodiment provides a click rate prediction device based on multi-level feature interaction, the device comprising:

[0118] The data set construction module is used to collect user feature data and historical behavior information to construct a training data set;

[0119] The xDeepFM model improvement module is used to add a second-order feature interaction layer between the feature embedding layer and the fully connected layer of the xDeepFM model to improve the xDeepFM model, and use the improved xDeepFM model as a click-through rate prediction model;

[0120] A model training module, used to train the click rate prediction model using the training data set;

[0121] The click rate prediction module is used to predict the click rate to be predicted using the trained click rate prediction model to obtain the prediction result.

[0122] The click-through rate prediction device based on multi-level feature interaction of this embodiment corresponds to the click-through rate prediction method based on multi-level feature interaction of the above-mentioned first embodiment; wherein, the functions implemented by each functional module in the click-through rate prediction device based on multi-level feature interaction of this embodiment correspond one-to-one to each process step in the click-through rate prediction method based on multi-level feature interaction of the first embodiment; therefore, they will not be repeated here.

[0123] Third embodiment

[0124] This embodiment provides an electronic device, which includes a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method of the first embodiment.

[0125] The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) and one or more memories, wherein the memory stores at least one instruction, and the instruction is loaded by the processor to execute the above method.

[0126] Fourth embodiment

[0127] This embodiment provides a computer-readable storage medium, which stores at least one instruction, and the instruction is loaded and executed by a processor to implement the method of the first embodiment. The computer-readable storage medium may be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein may be loaded by a processor in a terminal to execute the above method.

[0128] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0129] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0130] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0131] It should also be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0132] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for those skilled in the art, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A click rate prediction method based on multi-level feature interaction, characterized in that: include: Collecting user feature data and historical behavior information to build a training data set, wherein the feature data includes: user age, user gender, mobile phone model used by the user, and user location information; the historical behavior information includes: user browsing time and whether the user clicks on an advertisement; A second-order feature interaction layer is added between the feature embedding layer and the fully connected layer of the xDeepFM model to improve the xDeepFM model, and the improved xDeepFM model is used as the click rate prediction model; Using the training data set to train the click rate prediction model; Use the trained click rate prediction model to predict the click rate to be predicted and obtain the prediction result; The click rate prediction model includes a feature embedding layer, a compressed interaction network, a second-order feature interaction layer, a fully connected layer, and a prediction layer, which fully integrates low-order and high-order features, and reduces the number of parameters by multiplying the second-order feature vectors, thereby improving the feature combination capability of the model; wherein, The feature embedding layer is used to convert the high-dimensional sparse feature vector into a low-dimensional dense feature vector, and input the converted feature vector into the compression interaction network and the second-order feature interaction layer; The second-order feature interaction layer is used to multiply the vectors in the n domains in pairs to obtain n*(n-1) / 2 vectors, and add these vectors to obtain a low-dimensional feature vector y e , calculated as follows: in, Represents the element-wise product of two vectors, e i represents the embedding vector of the ith domain, e j represents the embedding vector of the jth domain, v i represents the weight of the ith domain, v j represents the weight of the jth domain; The second-order feature interaction layer inputs the obtained feature vector into the fully connected layer to learn high-order feature interactions. The calculation method is as follows: Where l is the number of the hidden layer, is a nonlinear activation function, is the output vector of the lth hidden layer, represents the output vector of the l-1th hidden layer, W (l) represents the weight matrix of the lth hidden layer, b (l) Represents the bias of the lth hidden layer; The prediction layer integrates linear regression, the compressed interaction network and the fully connected layer into one framework, and the calculation method is as follows: in, represents the output of the prediction layer, σ is the Sigmoid function, x represents the original feature value that has not been processed by the feature embedding layer, and y dnn ,y cin are the output values ​​of the fully connected layer and the compressed interaction network, respectively, linear , W dnn and W cin are the parameter models of linear regression, the fully connected layer and the compressed interaction network, respectively, and b dnn and b are bias parameters of the fully connected layer and the prediction layer, respectively; The click rate prediction method based on multi-level feature interaction also includes: The maximum support dimension parameter M is introduced in the one-hot encoding process of the click rate prediction model to prevent dimension explosion. The dictionary mapping relationship between the original data and the occurrence frequency is as follows: P[x i ]=N i ,i∈{1,2,...,K} Among them, x i is the ith different hash value, N i For x i The corresponding frequency, K is the total number of hash values; if K≤M, then just change x i The corresponding index takes the value of 1 in the one-hot encoding. If K>M, the frequency N needs to be i Perform secondary encoding and create a new mapping table until the dimension is less than M; P[x i ] represents the frequency of occurrence of the i-th feature; The click rate prediction method based on multi-level feature interaction also includes: In the training process of the click rate prediction model, the early stopping method is used to select the optimal number of iterations; wherein the expression of the early stopping method is as follows: Among them, E opt (t) and E va (t) are the best validation set error and test set error at iteration t; when the generalization loss GL(t) exceeds a certain threshold, training is stopped.

2. The click rate prediction method based on multi-level feature interaction as claimed in claim 1, characterized in that: The collecting of user feature data and historical behavior information to construct a training data set includes: Collect user's characteristic data and historical behavior information; The collected feature data and historical behavior information are preprocessed to extract continuous features and classification features respectively, and the continuous features are placed before the classification features to construct a training data set.

3. A click rate prediction device based on multi-level feature interaction, characterized in that: include: The data set construction module is used to collect the user's characteristic data and historical behavior information to construct a training data set, wherein the characteristic data includes: user age, user gender, user's mobile phone model and user location information; the historical behavior information includes: user's browsing time and whether the user clicks on an advertisement; The xDeepFM model improvement module is used to add a second-order feature interaction layer between the feature embedding layer and the fully connected layer of the xDeepFM model to improve the xDeepFM model, and use the improved xDeepFM model as a click-through rate prediction model; A model training module, used to train the click rate prediction model using the training data set; A click rate prediction module is used to predict the click rate to be predicted using a trained click rate prediction model to obtain a prediction result; The click rate prediction model includes a feature embedding layer, a compressed interaction network, a second-order feature interaction layer, a fully connected layer, and a prediction layer, which fully integrates low-order and high-order features, and reduces the number of parameters by multiplying the second-order feature vectors, thereby improving the feature combination capability of the model; wherein, The feature embedding layer is used to convert the high-dimensional sparse feature vector into a low-dimensional dense feature vector, and input the converted feature vector into the compression interaction network and the second-order feature interaction layer; The second-order feature interaction layer is used to multiply the vectors in the n domains in pairs to obtain n*(n-1) / 2 vectors, and add these vectors to obtain a low-dimensional feature vector y e , calculated as follows: in, Represents the element-wise product of two vectors, e i represents the embedding vector of the ith domain, e j represents the embedding vector of the jth domain, v i represents the weight of the ith domain, v j represents the weight of the jth domain; The second-order feature interaction layer inputs the obtained feature vector into the fully connected layer to learn high-order feature interactions. The calculation method is as follows: Where l is the number of the hidden layer, is a nonlinear activation function, is the output vector of the lth hidden layer, represents the output vector of the l-1th hidden layer, W (l) represents the weight matrix of the lth hidden layer, b (l) Represents the bias of the lth hidden layer; The prediction layer integrates linear regression, the compressed interaction network and the fully connected layer into one framework, and the calculation method is as follows: in, represents the output of the prediction layer, σ is the Sigmoid function, x represents the original feature value that has not been processed by the feature embedding layer, and y dnn ,y cin are the output values ​​of the fully connected layer and the compressed interaction network, respectively, linear , W dnn and W cin are the parameter models of linear regression, the fully connected layer and the compressed interaction network, respectively, and b dnn and b are bias parameters of the fully connected layer and the prediction layer, respectively; The click rate prediction device based on multi-level feature interaction also includes: The module for preventing dimension explosion is used to introduce a maximum support dimension parameter in the one-hot encoding process of the click rate prediction model to prevent dimension explosion. The dictionary mapping relationship between the original data and the occurrence frequency is as follows: P[x i ]=N i ,i∈{1,2,...,K} Among them, x i is the ith different hash value, N i For x i The corresponding frequency, K is the total number of hash values; if K≤M, then just change x i The corresponding index takes the value of 1 in the one-hot encoding. If K>M, the frequency N needs to be i Perform secondary encoding and create a new mapping table until the dimension is less than M; P[x i ] represents the frequency of occurrence of the i-th feature; The click rate prediction device based on multi-level feature interaction also includes: The optimal number of iterations selection module is used to use the early stopping method to select the optimal number of iterations during the training process of the click rate prediction model; wherein the expression of the early stopping method is as follows: Among them, E opt (t) and E va (t) are the best validation set error and test set error at iteration t; when the generalization loss GL(t) exceeds a certain threshold, training is stopped.

4. The click rate prediction device based on multi-level feature interaction as claimed in claim 3, characterized in that: The data set construction module is specifically used for: Collect user's characteristic data and historical behavior information; The collected feature data and historical behavior information are preprocessed to extract continuous features and classification features respectively, and the continuous features are placed before the classification features to construct a training data set.

Images