Click-through rate prediction method, device, computer equipment, and storage medium

Abstract

The present application relates to a click-through rate prediction method, apparatus, computer equipment, storage medium and computer program product. The method comprises: obtaining each coded discrete feature and coded continuous feature of the target user; performing a linear transformation on the output cross-feature matrix to obtain a cross-feature linear transformation matrix; weighting each embedded feature by feature importance to obtain each weighted embedded feature; performing an explicit low-order feature cross on each weighted embedded feature to obtain a low-order combination feature; performing an explicit high-order feature cross on each embedded feature to obtain a high-order combination feature; performing feature aggregation on the low-order combination feature, the high-order combination feature and the coded continuous feature to obtain an aggregated feature, and predicting the click-through rate of the target user based on the aggregated feature. The use of this method can improve the accuracy of click-through rate prediction.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a click-through rate prediction method, apparatus, computer equipment, storage medium, and computer program product. Background Art

[0002] With the development of big data technology, deep neural networks are often used to solve the problem of click-through rate prediction in recommendation systems. The input of a deep neural network typically includes user features and item features. These are fed into the input layer of the neural network. Then, multiple hidden layers perform nonlinear transformations and feature extraction, learning the interactions between the original features and ultimately outputting a predicted value: the probability that the user will click on the target item. In the process of click-through rate prediction, learning feature interactions is crucial to the prediction results. Because directly using the original features for prediction rarely yields optimal results, deep neural network models automatically learn the interactions between the various features and then use these cross-features to predict click-through rates, thereby improving the accuracy and personalization of recommendations.

[0003] Currently, an increasing number of deep learning models use explicit feature intersection to construct relationships between features. Explicit feature intersection refers to combining multiple features in a deep learning model, either manually or through mathematical formulas, to form new features that better describe the relationships between the data. For example, suppose we are predicting housing prices. If we find a relationship between population density and the number of rooms, we can create a new feature, multiply these two features, and use the resulting value as the feature intersection of the two features.

[0004] However, different features usually have different impacts on click-through rate prediction results. The main disadvantage of previous explicit feature crossing methods is that they pay more attention to high-order feature interactions and ignore low-order feature interactions, and the degree of attention to all features is the same. Therefore, after explicit feature crossing, invalid and insignificant features will still have a negative impact on the combined features obtained by explicit feature crossing, affecting the accuracy of the combined features, and thus reducing the accuracy of click-through rate prediction. Summary of the Invention

[0005] Based on this, it is necessary to provide a click-through rate prediction method, apparatus, computer device, computer-readable storage medium and computer program product that can improve the accuracy of click-through rate prediction based on explicit feature cross-talk to address the above technical problems.

[0006] In a first aspect, the present application provides a click-through rate prediction method. The method comprises:

[0007] Obtaining each coded discrete feature and coded continuous feature of the target user, wherein the coded discrete features are obtained by feature encoding discrete features in the user features and item features, and the coded continuous features are obtained by feature encoding continuous features in the user features and item features;

[0008] Converting each of the encoded discrete features into a corresponding embedded feature;

[0009] Performing feature importance weighting on each of the embedded features to obtain weighted embedded features;

[0010] Performing explicit low-order feature intersection on each of the weighted embedded features to obtain a low-order combined feature;

[0011] Performing explicit high-order feature intersection on each of the embedded features to obtain a high-order combined feature;

[0012] The low-order combined features, the high-order combined features, and the coded continuous features are subjected to feature aggregation to obtain aggregated features, and click-through rate prediction is performed on the target user based on the aggregated features.

[0013] In one embodiment, weighting the importance of each embedded feature to obtain each weighted embedded feature includes:

[0014] Predicting an attention score corresponding to each of the embedded features, wherein the attention score is used to characterize the importance of the embedded feature; and weighting each of the embedded features according to each of the attention scores to obtain each weighted embedded feature.

[0015] In one embodiment, predicting the attention score corresponding to each of the embedded features includes:

[0016] The feature information in each embedded feature is compressed respectively to obtain each information compression feature value; the feature vector composed of each information compression feature value is fully connected a preset number of times to obtain an attention score vector, wherein the attention score vector includes the attention score of each embedded feature.

[0017] In one embodiment, performing explicit high-order feature intersection on each of the embedded features to obtain a high-order combined feature includes:

[0018] The embedded feature matrices composed of each of the embedded features are linearly transformed into a query weight matrix, a key weight matrix and a value matrix respectively; the query weight matrix, the key weight matrix and the value matrix are subjected to multi-head attention transformation to obtain an output cross feature matrix; the output cross feature matrix is linearly transformed to obtain a cross feature linear transformation matrix; the cross feature linear transformation matrix is used as a new query weight matrix, and the execution step is returned to: the query weight matrix, the key weight matrix and the value matrix are subjected to multi-head attention transformation to obtain an output cross feature matrix, until the number of feature crosses corresponding to the output cross feature matrix reaches a preset number, and the output cross feature matrix is output as a high-order combined feature.

[0019] In one embodiment, performing a multi-head attention transformation on the query weight matrix, the key weight matrix, and the value matrix to obtain an output cross-feature matrix includes:

[0020] The query weight matrix, the key weight matrix and the value matrix are split respectively to obtain a preset number of sub-matrix sets, wherein the sub-matrix sets include the split query matrix of the query weight matrix, the split key matrix of the key weight matrix and the split value matrix of the value matrix; according to the matrix product between the split query matrix and the split key matrix in each sub-matrix set, the attention weight matrix corresponding to each split value matrix is determined respectively; according to each attention weight matrix, the feature importance of each split value matrix is weighted to obtain each weighted split value matrix; each weighted split value matrix is connected to obtain an output cross feature matrix.

[0021] In one embodiment, the step of performing feature aggregation on the low-order combined features, the high-order combined features, and the coded continuous features to obtain aggregated features includes:

[0022] The low-order combined features are expanded into a one-dimensional vector to obtain a first combined feature vector; the high-order combined features are expanded into a one-dimensional vector to obtain a second combined feature vector; the first combined feature vector, the second combined feature vector and the encoded continuous features are fused to obtain an aggregated feature.

[0023] In a second aspect, the present application further provides a click-through rate prediction device. The device comprises:

[0024] a feature processing module, configured to obtain coded discrete features and coded continuous features of the target user, wherein the coded discrete features are obtained by feature encoding discrete features among the user features and item features, and the coded continuous features are obtained by feature encoding continuous features among the user features and item features;

[0025] An embedding feature conversion module, configured to convert each of the encoded discrete features into a corresponding embedding feature;

[0026] A feature importance weighting module, configured to weight the importance of each embedded feature to obtain weighted embedded features;

[0027] An explicit low-order feature crossover module, configured to perform explicit low-order feature crossover on each of the weighted embedded features to obtain a low-order combined feature;

[0028] An explicit high-order feature intersection module, configured to perform explicit high-order feature intersection on each of the embedded features to obtain a high-order combined feature;

[0029] The prediction module is used to perform feature aggregation on the low-order combined features, the high-order combined features and the coded continuous features to obtain aggregated features, and perform click-through rate prediction on the target user based on the aggregated features.

[0030] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0031] Obtain each coded discrete feature and coded continuous feature of the target user, wherein the coded discrete feature is obtained by feature encoding the discrete features in the user features and the item features, and the coded continuous feature is obtained by feature encoding the continuous features in the user features and the item features; convert each coded discrete feature into a corresponding embedded feature; weight each embedded feature by feature importance to obtain each weighted embedded feature; perform explicit low-order feature crossover on each weighted embedded feature to obtain a low-order combination feature; perform explicit high-order feature crossover on each embedded feature to obtain a high-order combination feature; perform feature aggregation on the low-order combination features, the high-order combination features and the coded continuous features to obtain an aggregated feature, and predict the click rate of the target user based on the aggregated feature.

[0032] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0033] Obtain each coded discrete feature and coded continuous feature of the target user, wherein the coded discrete feature is obtained by feature encoding the discrete features in the user features and the item features, and the coded continuous feature is obtained by feature encoding the continuous features in the user features and the item features; convert each coded discrete feature into a corresponding embedded feature; weight each embedded feature by feature importance to obtain each weighted embedded feature; perform explicit low-order feature crossover on each weighted embedded feature to obtain a low-order combination feature; perform explicit high-order feature crossover on each embedded feature to obtain a high-order combination feature; perform feature aggregation on the low-order combination features, the high-order combination features and the coded continuous features to obtain an aggregated feature, and predict the click rate of the target user based on the aggregated feature.

[0034] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0035] Obtain each coded discrete feature and coded continuous feature of the target user, wherein the coded discrete feature is obtained by feature encoding the discrete features in the user features and the item features, and the coded continuous feature is obtained by feature encoding the continuous features in the user features and the item features; convert each coded discrete feature into a corresponding embedded feature; weight each embedded feature by feature importance to obtain each weighted embedded feature; perform explicit low-order feature crossover on each weighted embedded feature to obtain a low-order combination feature; perform explicit high-order feature crossover on each embedded feature to obtain a high-order combination feature; perform feature aggregation on the low-order combination features, the high-order combination features and the coded continuous features to obtain an aggregated feature, and predict the click rate of the target user based on the aggregated feature.

[0036] The above-mentioned click-through rate prediction method, device, computer equipment, storage medium and computer program product obtain the coded discrete features and coded continuous features of the target user, wherein the coded discrete features are obtained by feature encoding the discrete features in the user features and the item features, and the coded continuous features are obtained by feature encoding the continuous features in the user features and the item features; each of the coded discrete features is converted into a corresponding embedded feature, so that the original discrete features in the user features and the item features can be reduced to the corresponding embedded features; each of the embedded features is weighted by feature importance to obtain each weighted embedded feature; each of the weighted embedded features is subjected to explicit low-order feature crossover to obtain a low-order combined feature, so that after weighting according to the importance of each embedded feature, explicit low-order feature crossover is performed. The high-order feature crossover can suppress invalid and insignificant features and strengthen important high-frequency features, so that the low-order combination features obtained by explicit low-order feature crossover are more accurate; then, explicit high-order feature crossover is performed on each of the embedded features to obtain high-order combination features; the low-order combination features, the high-order combination features and the encoded continuous features are subjected to feature aggregation to obtain aggregated features, so that the low-order feature combination, the high-order feature combination and the encoded continuous features can be combined to make the final aggregated features richer, and more accurate low-order combination features can also help to improve the accuracy of the aggregated features, and then, based on the aggregated features, the click-through rate of the target user is predicted, which can achieve click-through rate prediction based on richer and more accurate features, thereby improving the accuracy of click-through rate prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 1 is a flow chart of a click rate prediction method according to an embodiment;

[0038] Figure 2 FIG. 1 is a schematic diagram of a process for weighting the importance of each embedded feature in one embodiment;

[0039] Figure 3 FIG. 1 is a flow chart of performing explicit high-order feature intersection on each embedded feature in another embodiment;

[0040] Figure 4 Schematic diagram of a process for performing multi-head attention transformation in another embodiment;

[0041] Figure 5 is a structural block diagram of a click rate prediction device in one embodiment;

[0042] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0044] In one embodiment, Figure 1 As shown, a click-through rate prediction method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, or to a system including a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0045] Step 202: Obtain the coded discrete features and coded continuous features of the target user, wherein the coded discrete features are obtained by feature encoding the discrete features in the user features and the item features, and the coded continuous features are obtained by feature encoding the continuous features in the user features and the item features.

[0046] The click-through rate prediction method in this embodiment can be used in a recommendation system, which can be used in a message recommendation scenario or an item recommendation scenario. The message can be an advertisement, a text message, or an image, etc. The predicted click-through rate can be used to set the message recommendation strategy or item recommendation strategy in the recommendation system. For example, the recommendation frequency of high-click-through rate messages or high-click-through rate items can be increased, and low-click-through rate messages or items can be avoided from being recommended to the target user. User features are characteristics of the user, such as age or gender, and item features are characteristics of the recommended item, such as price or category. Discrete features are features whose characteristic values are discretely distributed in user features and item features, such as category or gender, and continuous features are features whose characteristic values are continuously distributed in user features and item features, such as price or age. Feature encoding methods can include one-hot encoding, multi-hot encoding, or normalization.

[0047] As an example, step 202 includes: obtaining each discrete feature and continuous feature in the user feature and the item feature; performing feature encoding on each discrete feature to obtain each encoded discrete feature, wherein the feature encoding method for the discrete feature can be one-hot encoding or multi-hot encoding; performing standardization on the continuous feature to normalize the feature value in the continuous feature to a preset value range, to obtain an encoded continuous feature, wherein the preset value range can be 0 to 1.

[0048] Step 204: convert each encoded discrete feature into a corresponding embedded feature.

[0049] Among them, this embodiment is provided with an embedding layer, which can be an Embedding layer, used to convert high-dimensional sparse features into low-dimensional dense features to achieve feature dimensionality reduction; the encoded discrete features can be feature vectors, and the number can be multiple.

[0050] As an example, based on the embedding layer, the encoded discrete features are converted into low-dimensional dense features to obtain the embedded features corresponding to the encoded discrete features.

[0051] As an example, each encoded discrete feature has a corresponding Embedding matrix in the Embedding layer, and the Embedding matrix can be a summary matrix of the Embedding vectors corresponding to all feature values in the discrete features.

[0052] As an example, the specific calculation process corresponding to step 204 is as follows:

[0053] e i =m i E i

[0054] Among them, m i is the feature vector of the i-th discrete feature, which can be a one-hot vector or a multi-hot vector; E i is the Embedding matrix of the i-th discrete feature, e i is the embedding feature of the i-th discrete feature, specifically the Embedding vector.

[0055] Step 206: Weight the feature importance of each embedded feature to obtain each weighted embedded feature.

[0056] This embodiment is provided with a feature selection layer, which is used to perform feature selection on each embedded feature to strengthen the features with high feature importance among the embedded features and suppress the features with low feature importance among the embedded features. Among them, the embedded features with higher feature importance have a greater contribution to the click-through rate prediction, that is, they are more likely to affect the results of the click-through rate prediction. The embedded features with lower feature importance have a smaller contribution to the click-through rate prediction, that is, they are less likely to affect the results of the click-through rate prediction.

[0057] As an example, step 206 includes: based on the feature selection layer, predicting the attention score corresponding to each embedded feature; according to each attention score, weighting the feature importance of each embedded feature to obtain each weighted embedded feature, wherein the higher the attention score, the higher the feature importance, and the correspondingly greater the weight when weighting the feature importance.

[0058] In step 208, explicit low-order feature crossover is performed on each weighted embedded feature to obtain a low-order combined feature.

[0059] Among them, explicit feature crossing refers to combining multiple features manually or through mathematical formulas in a deep learning model to form new features to better describe the relationship between data; explicit low-order feature crossing refers to explicit feature crossing in which the number of feature crossings is less than a preset number threshold. As an example, the preset number threshold can be 2.

[0060] As an example, step 208 includes: performing pairwise Hadamard products on each weighted embedded feature to obtain an output matrix of explicit low-order feature intersections; and using the output matrix of explicit low-order feature intersections as low-order combined features.

[0061] In step 210 , explicit high-order feature crossover is performed on each embedded feature to obtain a high-order combined feature.

[0062] In this embodiment, a multi-head attention module is provided, which is used to perform explicit high-order feature intersection on each embedded feature. Explicit high-order feature intersection refers to explicit feature intersection in which the number of feature intersections is not less than a preset number threshold. As an example, the preset number threshold can be 2.

[0063] As an example, step 210 includes: obtaining an embedded feature matrix composed of each embedded feature, and using the embedded feature matrix as the input feature matrix; based on the multi-head attention module, performing a multi-head attention transformation based on feature importance on the embedded feature matrix to obtain an output cross feature matrix; using the output cross feature matrix and the embedded feature matrix together as the input feature matrix, and returning to the execution step: based on the multi-head attention module, performing a multi-head attention transformation based on feature importance on the embedded feature matrix to obtain an output cross feature matrix, until the number of feature crosses corresponding to the output cross feature matrix reaches a preset number, and using the input cross feature matrix as a high-order combined feature, wherein, in this embodiment, each time a multi-head attention transformation is performed, it is considered that the number of feature crosses increases once, that is, the number of multi-head attention transformations is equal to the number of feature crosses.

[0064] In step 212 , the low-order combined features, the high-order combined features, and the coded continuous features are aggregated to obtain aggregated features, and click-through rate prediction is performed for the target user based on the aggregated features.

[0065] As an example, low-order combined features, high-order combined features, and encoded continuous features are aggregated to obtain aggregated features. Based on a preset multi-layer perceptron network and a preset normalization module, the aggregated features are converted into click probabilities corresponding to target users. This click probability is the target user's click-through rate for recommended items or messages. The preset normalization module can be a Sigmoid function, and the feature aggregation method can be concatenation, weighted averaging, or accumulation, etc., which are not limited here. The preset multi-layer perceptron network is composed of multiple layers of a fully connected network.

[0066] As an example, the specific calculation process for predicting the click rate of target users based on aggregated features is as follows:

[0067]

[0068] Among them, W l is the weight matrix of the fully connected network at layer l, b l is the bias vector of the lth layer fully connected network, o l is the output vector of the l-th layer fully connected network, o l The click rate of target users on recommended items or recommended messages.

[0069] As an example, low-order combined features, high-order combined features, and coded continuous features are aggregated to obtain aggregated features, including:

[0070] The low-order combined features are expanded into a one-dimensional vector to obtain a first combined feature vector; the high-order combined features are expanded into a one-dimensional vector to obtain a second combined feature vector; the first combined feature vector, the second combined feature vector and the encoded continuous features are fused to obtain an aggregated feature.

[0071] Specifically, each eigenvalue in the low-order combined feature is expanded into a one-dimensional vector to obtain a first combined feature vector; each eigenvalue in the high-order combined feature is expanded into a one-dimensional vector to obtain a second combined feature vector; the first combined feature vector, the second combined feature vector and the encoded continuous feature are spliced into an aggregated feature.

[0072] In the above-mentioned click-through rate prediction method, the target user's coded discrete features and coded continuous features are obtained, wherein the coded discrete features are obtained by feature encoding the discrete features in the user features and the item features, and the coded continuous features are obtained by feature encoding the continuous features in the user features and the item features; the output cross-feature matrix is linearly transformed to obtain a cross-feature linear transformation matrix, so that the original discrete features in the user features and the item features can be reduced to corresponding embedded features; each embedded feature is weighted by feature importance to obtain each weighted embedded feature; each weighted embedded feature is explicitly cross-linked with a low-order feature to obtain a low-order combined feature, so that the explicit low-order feature cross-link is performed after weighting according to the importance of each embedded feature. , which can suppress invalid and insignificant features and strengthen important high-frequency features, so that the low-order combination features obtained by crossing explicit low-order features are more accurate; then, explicit high-order feature crossing is performed on each embedded feature to obtain high-order combination features; low-order combination features, high-order combination features and encoded continuous features are aggregated to obtain aggregate features, so that low-order feature combinations, high-order feature combinations and encoded continuous features can be combined to make the final aggregate features richer, and more accurate low-order combination features can also help improve the accuracy of aggregate features, and then, based on the aggregate features, the click-through rate of the target user is predicted, which can achieve click-through rate prediction based on richer and more accurate features, thereby improving the accuracy of click-through rate prediction.

[0073] In one embodiment, Figure 2 As shown in Figure 2, each embedding feature is weighted by its importance to obtain each weighted embedding feature, including:

[0074] Step 302: predict the attention score corresponding to each embedded feature, where the attention score is used to represent the importance of the embedded feature.

[0075] Among them, the above-mentioned feature selection layer includes a feature compression layer and a fully connected network layer. The feature compression layer is used to compress each embedded feature into a feature vector. The fully connected network layer is used to convert the feature vector output by the feature compression layer into an attention score vector. The attention score vector includes the attention score corresponding to each embedded feature; the attention score is used to characterize the importance of the embedded feature. The higher the attention score, the higher the feature importance of the embedded feature, and the lower the attention score, the lower the feature importance of the embedded feature.

[0076] As an example, step 302 includes: based on the feature compression layer, compressing each embedded feature into a feature vector; based on the fully connected network layer, converting the feature vector output by the feature compression layer into an attention score vector, wherein the attention score vector includes the attention score corresponding to each embedded feature.

[0077] In step 304, each embedded feature is weighted according to each attention score to obtain each weighted embedded feature.

[0078] As an example, step 304 includes: respectively calculating the product between each attention score and the embedded feature corresponding to each attention score to obtain each weighted embedded feature. Wherein, both the embedded feature and the weighted embedded feature can be feature vectors.

[0079] As an example, the calculation process of weighting each embedding feature according to each attention score to obtain each weighted embedding feature is as follows:

[0080] V=A*E=[a1*e1, a2*e2,…,a n *e n ]=[v1,v2,…,v n ]

[0081] Among them, v n is the weighted embedding feature corresponding to the nth embedding feature, e n is the nth embedding feature, a n is the attention score corresponding to the nth embedded feature, A is the attention score vector, E is the vector matrix composed of n embedded features, and V is the vector matrix composed of n weighted embedded features.

[0082] In this embodiment, the attention score corresponding to each embedded feature is predicted, wherein the attention score is used to characterize the importance of the embedded feature. According to each attention score, each embedded feature is weighted to obtain each weighted embedded feature. According to the feature importance of each embedded feature, each embedded feature can be weighted. This can strengthen the embedded features with high feature importance and suppress the embedded features with low feature importance, thereby making the low-order combined features obtained by cross-pollinating low-order features based on each weighted embedded feature more accurate.

[0083] In one embodiment, predicting the attention score corresponding to each embedded feature includes:

[0084] The feature information in each embedded feature is compressed respectively to obtain the compressed feature value of each information; the feature vector composed of the compressed feature value of each information is fully connected a preset number of times to obtain an attention score vector, wherein the attention score vector includes the attention score of each embedded feature.

[0085] Specifically, by fusing the eigenvalues in each embedded feature respectively and compressing the feature information in each embedded feature respectively, information compression eigenvalues are obtained, wherein the fusion method can be averaged; the eigenvector composed of each information compression eigenvalue is passed through a fully connected network with a preset number of layers, and the eigenvector composed of each information compression eigenvalue is fully connected a preset number of times to obtain an attention score vector, wherein the attention score vector includes the attention score of each embedded feature.

[0086] This embodiment compresses the feature information in each embedded feature separately to obtain each information compression feature value, and can summarize the feature information in each embedded feature into a feature vector composed of each information compression feature value. Then, the feature vector composed of each information compression feature value is fully connected a preset number of times to generate an attention score vector corresponding to each embedded feature, thereby achieving an accurate assessment of the feature importance of each embedded feature.

[0087] In one embodiment, Figure 3 As shown in the figure, explicit high-order feature crossover is performed on each embedded feature to obtain high-order combined features, including:

[0088] Step 402: linearly transform the embedded feature matrix composed of each embedded feature into a query weight matrix, a key weight matrix, and a value matrix respectively;

[0089] In this embodiment, a linear transformation layer and a multi-head attention transformation layer are provided to form a multi-head attention module; the linear transformation layer includes a first linear transformation parameter, a second linear transformation parameter and a third linear transformation parameter, the first linear transformation parameter is used to transform the embedded feature matrix into a query weight matrix, the second linear transformation parameter is used to transform the embedded feature matrix into a key weight matrix, and the third linear transformation parameter is used to transform the embedded feature matrix into a value matrix. The query weight matrix and the key weight matrix are used together to calculate the feature cross weight when performing the multi-head attention transformation, and the value matrix represents the original feature vector of each embedded feature.

[0090] As an example, step 402 includes: performing a linear transformation on the embedded feature matrix based on a first linear transformation parameter to obtain a query weight matrix; performing a linear transformation on the embedded feature matrix based on a second linear transformation parameter to obtain a key weight matrix; performing a linear transformation on the embedded feature matrix based on a third linear transformation parameter to obtain a value matrix.

[0091] As an example, the calculation process in the linear transformation layer is as follows:

[0092]

[0093] Among them, Q is the query weight matrix, K is the key weight matrix, V is the value matrix, and W q is the first linear transformation parameter, Wk is the second linear transformation parameter, W v is the third linear transformation parameter, and E is the embedded feature matrix composed of each embedded feature.

[0094] Step 404: Perform a multi-head attention transformation on the query weight matrix, the key weight matrix, and the value matrix to obtain an output cross-feature matrix.

[0095] As an example, step 404 includes: based on the multi-head attention transformation layer, splitting the query weight matrix, the key weight matrix and the value matrix into a preset number of sub-matrix sets, wherein the sub-matrix set includes the split query matrix of the query weight matrix, the split key matrix of the key weight matrix and the split value matrix of the value matrix; according to the split query matrix and the split key matrix in each sub-matrix set, calculating the attention weight matrix of the split value matrix in each sub-matrix set, wherein the attention weight matrix includes the attention weight of each eigenvalue in the split value matrix, which is used to perform a weighted operation on each eigenvalue in the split value matrix to enhance the eigenvalues with high feature importance in the split value matrix and suppress the eigenvalues with low feature importance in the split value matrix; according to each attention weight matrix, weighting the feature importance of each split value matrix to obtain each weighted split value matrix; connecting each weighted split value matrix into the output cross feature matrix of the multi-head attention transformation layer.

[0096] Step 406, performing a linear transformation on the output cross-feature matrix to obtain a cross-feature linear transformation matrix;

[0097] As an example, step 406 includes: performing a linear transformation on the output cross-feature matrix based on the first linear transformation parameter to obtain a cross-feature linear transformation matrix.

[0098] As an example, the specific calculation process of step 406 is as follows:

[0099] Q′=(W q H2) T

[0100] Among them, Q′ is the cross-feature linear transformation matrix, W q is the first linear transformation parameter, and H2 is the output cross-feature matrix.

[0101] Step 408: Use the cross-feature linear transformation matrix as the new query weight matrix and return to the execution step: perform multi-head attention transformation on the query weight matrix, key weight matrix and value matrix to obtain the output cross-feature matrix, until the number of feature crosses corresponding to the output cross-feature matrix reaches a preset number, and output the output cross-feature matrix as a high-order combined feature.

[0102] Among them, the number of feature crossovers can be the number of multi-head attention changes, and the preset number can be set to be greater than or equal to 2.

[0103] As an example, a multi-head attention transformation is performed on the query weight matrix, the key weight matrix, and the value matrix to obtain the output cross-feature matrix, including:

[0104] The query weight matrix, key weight matrix and value matrix are split respectively to obtain a preset number of sub-matrix sets, wherein the sub-matrix sets include the split query matrix of the query weight matrix, the split key matrix of the key weight matrix and the split value matrix of the value matrix; according to the matrix product between the split query matrix and the split key matrix in each sub-matrix set, the attention weight matrix corresponding to each split value matrix is determined respectively; according to each attention weight matrix, the feature importance of each split value matrix is weighted respectively to obtain each weighted split value matrix; each weighted split value matrix is connected to obtain an output cross feature matrix.

[0105] Specifically, the query weight matrix, the key weight matrix and the value matrix are split respectively to obtain a preset number of sub-matrix sets, wherein the sub-matrix sets include the split query matrix of the query weight matrix, the split key matrix of the key weight matrix and the split value matrix of the value matrix; the matrix products between the split query matrix and the split key matrix in each sub-matrix set are calculated respectively to obtain each similarity matrix, wherein the similarity matrix includes at least a similarity vector, which is used to characterize the similarity between the original feature and other features in a value matrix; each similarity matrix is normalized respectively to obtain each attention weight matrix, wherein each row in the attention weight matrix includes the attention score of each original feature to other features, which can characterize the interaction information between different features and can be normalized by the Softmax function; each attention weight matrix is matrix multiplied with the split value matrix corresponding to each attention weight matrix to weight the feature importance of each split value matrix respectively to obtain each weighted split value matrix; each weighted split value matrix is connected to form an output cross feature matrix.

[0106] As an example, suppose the query weight matrix is Q∈R n*d′ , the key weight matrix is K∈R n*d′ , the value matrix is V∈R n*d′ , where n is the number of discrete features, d′ is the dimension of the feature vectors in Q, K, and V, and then Q, K, and V are split into m heads, there will be m sub-matrix sets, as well as The split query matrix, split key matrix, and split value matrix in the i-th head (i-th submatrix set) are respectively. According to the matrix product between the split query matrix and the split key matrix in each submatrix set, the calculation process of determining the attention weight matrix corresponding to each split value matrix is as follows:

[0107]

[0108] Among them, S i is the matrix product between the split query matrix of the i-th head and the split key matrix, that is, the similarity matrix of the i-th head, Q i is the split query matrix of the i-th head, K i is the split key matrix of the i-th head, A i is the attention weight matrix of the i-th head; then, according to each attention weight matrix, each split value matrix is weighted by feature importance to obtain each weighted split value matrix, and each weighted split value matrix is connected to obtain the output cross feature matrix. The specific calculation process is as follows:

[0109]

[0110] Among them, H i is the weighted split value matrix of the i-th head, H2 is the output cross feature matrix obtained by the second-order feature cross, that is, the output cross feature matrix after a multi-head attention transformation, Indicates a connection.

[0111] As an example, Figure 4 As shown, Figure 4 This is a flow chart of multi-head attention transformation in this embodiment ( Figure 4 The attention head in is 1, that is, the query weight matrix, key weight matrix and value matrix are not split), where H l-1 is the output cross feature matrix obtained from the previous round of multi-head attention transformation, E is the embedding feature matrix, and W q is the first linear transformation parameter, W k is the second linear transformation parameter, W v is the third linear transformation parameter, Q is the query weight matrix, K is the key weight matrix, V is the value matrix, S is the similarity matrix, Dot Product represents the process of generating the similarity matrix in the above embodiment, A is the attention weight matrix, H l is the output cross feature matrix obtained by this round of multi-head attention transformation, and Matrix Multiplication represents matrix multiplication.

[0112] In this embodiment, the embedded feature matrix composed of each embedded feature is linearly transformed into a query weight matrix, a key weight matrix and a value matrix through a linear transformation layer, and then the query weight matrix, the key weight matrix and the value matrix are split respectively to obtain a preset number of sub-matrix sets, wherein the sub-matrix set includes a split query matrix of the query weight matrix, a split key matrix of the key weight matrix and a split value matrix of the value matrix; according to the matrix product between the split query matrix and the split key matrix in each sub-matrix set, the attention weight matrix corresponding to each split value matrix is determined respectively, so as to obtain the attention score of each embedded feature to other embedded features.

[0113] Furthermore, this embodiment also weights the feature importance of each split value matrix according to each attention weight matrix to obtain each weighted split value matrix, connects each weighted split value matrix to obtain an output cross feature matrix. Since each weighted split value matrix can represent the feature interaction relationship between the embedded features analyzed from one angle, the output cross feature matrix can represent the feature interaction relationship between the embedded features analyzed from multiple angles, and realizes the second-order feature intersection between the embedded features from multiple angles, which can improve the accuracy of the obtained cross features, and then performs a linear transformation on the output cross feature matrix to obtain a cross feature linear transformation matrix; the cross feature linear transformation matrix is used as a new query weight matrix, and returns to the execution step: multi-head attention transformation is performed on the query weight matrix, the key weight matrix and the value matrix to obtain the output cross feature matrix, until the number of feature intersections corresponding to the output cross feature matrix reaches a preset number, and higher-order feature intersections can be further achieved on the basis of the second-order feature intersection, and finally a high-order combination feature is obtained. Since each multi-head attention transformation is a feature intersection between each feature from multiple angles, the accuracy of the obtained high-order combination feature can be improved.

[0114] In one embodiment, the click-through rate prediction method of this embodiment includes: obtaining each discrete feature and continuous feature in user features and item features; performing feature encoding on each discrete feature to obtain each encoded discrete feature, wherein the feature encoding method for discrete features can be one-hot encoding or multi-hot encoding; performing standardization on continuous features to normalize the feature values in continuous features to a preset value range to obtain encoded continuous features, wherein the preset value range can be 0 to 1; based on the embedding layer, converting the encoded discrete features into low-dimensional dense features to obtain embedded features corresponding to the encoded discrete features; by fusing the feature values in each embedded feature, respectively compressing the feature information in each embedded feature to obtain information compression feature values, wherein the fusion method The formula can be averaged; the feature vector composed of each information compression eigenvalue is fully connected for a preset number of times to obtain an attention score vector, wherein the attention score vector includes the attention score of each embedded feature; the product between each attention score and the embedded feature corresponding to each attention score is calculated respectively to obtain each weighted embedded feature, wherein both the embedded feature and the weighted embedded feature can be a feature vector; the Hadamard product is performed on each weighted embedded feature to obtain the output matrix of the explicit low-order feature intersection; the output matrix of the explicit low-order feature intersection is used as the low-order combination feature to realize the explicit low-order feature intersection based on the attention mechanism, so that invalid and small-effect features can be suppressed during the explicit low-order feature intersection, and important high-frequency features can be strengthened, thereby improving the accuracy of the low-order combination features obtained by the explicit low-order feature intersection.

[0115] Then, the embedded feature matrix is linearly transformed to obtain a query weight matrix; based on the second linear transformation parameter, the embedded feature matrix is linearly transformed to obtain a key weight matrix; based on the third linear transformation parameter, the embedded feature matrix is linearly transformed to obtain a value matrix; the query weight matrix, the key weight matrix and the value matrix are split respectively to obtain a preset number of sub-matrix sets, wherein the sub-matrix set includes a split query matrix of the query weight matrix, a split key matrix of the key weight matrix and a split value matrix of the value matrix; the matrix product between the split query matrix and the split key matrix in each sub-matrix set is calculated respectively to obtain each similarity matrix, wherein the similarity matrix includes at least a similarity vector, which is used to characterize the similarity between the original feature and other features in a value matrix; each similarity matrix is normalized respectively to obtain each attention weight matrix, wherein each row in the attention weight matrix includes the attention score of each original feature to other features, which can characterize the interaction information between different features and can be normalized by the Softmax function.

[0116] By performing matrix multiplication on each attention weight matrix and the split value matrix corresponding to each attention weight matrix, the feature importance of each split value matrix is weighted to obtain each weighted split value matrix; each weighted split value matrix is connected to form an output cross feature matrix; the output cross feature matrix is input into the linear transformation layer, and the output cross feature matrix is linearly transformed based on the first linear transformation parameter to obtain a cross feature linear transformation matrix; the cross feature linear transformation matrix is used as the new query weight matrix, and the execution step is returned: multi-head attention transformation is performed on the query weight matrix, the key weight matrix and the value matrix to obtain the output cross feature matrix, until the number of feature crosses corresponding to the output cross feature matrix reaches a preset number, and the output cross feature matrix is output as a high-order combination feature, thereby realizing explicit high-order feature crossover based on the attention mechanism. In this way, invalid and small-effect features can be suppressed during explicit high-order feature crossover, and important high-frequency features can be strengthened, thereby improving the accuracy of the high-order combination features obtained by explicit high-order feature crossover.

[0117] Then, each eigenvalue in the low-order combined feature is expanded into a one-dimensional vector to obtain a first combined feature vector; each eigenvalue in the high-order combined feature is expanded into a one-dimensional vector to obtain a second combined feature vector; the first combined feature vector, the second combined feature vector and the encoded continuous feature are spliced into an aggregated feature; the aggregated feature is converted into a click probability corresponding to the target user by passing the aggregated feature through a preset multi-layer perceptron network and a preset normalization module. The click probability is the click rate of the target user on the recommended item or recommended message. Since the aggregated feature is obtained by feature aggregation of low-order combined features, high-order combined features and encoded continuous features, the accuracy of the aggregated feature will also be higher. Based on this, this embodiment can use the attention mechanism to take into account the influence of different features on the prediction results, while combining explicit high-order feature cross and explicit low-order feature cross to construct a more accurate aggregated feature for click-through rate prediction, thereby improving the accuracy of click-through rate prediction.

[0118] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0119] Based on the same inventive concept, embodiments of the present application also provide a click-through rate prediction device for implementing the aforementioned click-through rate prediction method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the click-through rate prediction device provided below can be found in the above-mentioned limitations of the click-through rate prediction method and will not be repeated here.

[0120] In one embodiment, Figure 5 As shown, a click-through rate prediction device is provided, comprising: a feature processing module 502, an embedded feature conversion module 504, a feature importance weighting module 506, an explicit low-order feature cross module 508, an explicit high-order feature cross module 510 and a prediction module 512, wherein:

[0121] A feature processing module 502 is configured to obtain coded discrete features and coded continuous features of the target user, wherein the coded discrete features are obtained by feature encoding discrete features in the user features and item features, and the coded continuous features are obtained by feature encoding continuous features in the user features and item features;

[0122] An embedded feature conversion module 504 is used to convert each of the encoded discrete features into a corresponding embedded feature;

[0123] A feature importance weighting module 506 is used to weight the feature importance of each embedded feature to obtain each weighted embedded feature;

[0124] An explicit low-order feature crossover module 508 is configured to perform explicit low-order feature crossover on each of the weighted embedded features to obtain a low-order combined feature;

[0125] An explicit high-order feature intersection module 510 is used to perform explicit high-order feature intersection on each of the embedded features to obtain a high-order combined feature;

[0126] The prediction module 512 is configured to perform feature aggregation on the low-order combined features, the high-order combined features, and the coded continuous features to obtain aggregated features, and perform click-through rate prediction for the target user based on the aggregated features.

[0127] In one embodiment, the feature importance weighting module 506 is further configured to:

[0128] Predicting an attention score corresponding to each of the embedded features, wherein the attention score is used to characterize the importance of the embedded feature; and weighting each of the embedded features according to each of the attention scores to obtain each weighted embedded feature.

[0129] In one embodiment, the feature importance weighting module 506 is further configured to:

[0130] The feature information in each embedded feature is compressed respectively to obtain each information compression feature value; the feature vector composed of each information compression feature value is fully connected a preset number of times to obtain an attention score vector, wherein the attention score vector includes the attention score of each embedded feature.

[0131] In one embodiment, the explicit high-order feature intersection module 510 is further configured to:

[0132] The embedded feature matrices composed of each of the embedded features are linearly transformed into a query weight matrix, a key weight matrix and a value matrix respectively; the query weight matrix, the key weight matrix and the value matrix are subjected to multi-head attention transformation to obtain an output cross feature matrix; the output cross feature matrix is linearly transformed to obtain a cross feature linear transformation matrix; the cross feature linear transformation matrix is used as a new query weight matrix, and the execution step is returned to: the query weight matrix, the key weight matrix and the value matrix are subjected to multi-head attention transformation to obtain an output cross feature matrix, until the number of feature crosses corresponding to the output cross feature matrix reaches a preset number, and the output cross feature matrix is output as a high-order combined feature.

[0133] In one embodiment, the explicit high-order feature intersection module 510 is further configured to:

[0134] The query weight matrix, the key weight matrix and the value matrix are split respectively to obtain a preset number of sub-matrix sets, wherein the sub-matrix sets include the split query matrix of the query weight matrix, the split key matrix of the key weight matrix and the split value matrix of the value matrix; according to the matrix product between the split query matrix and the split key matrix in each sub-matrix set, the attention weight matrix corresponding to each split value matrix is determined respectively; according to each attention weight matrix, the feature importance of each split value matrix is weighted to obtain each weighted split value matrix; each weighted split value matrix is connected to obtain an output cross feature matrix.

[0135] In one embodiment, the prediction module 512 is further configured to:

[0136] The low-order combined features are expanded into a one-dimensional vector to obtain a first combined feature vector; the high-order combined features are expanded into a one-dimensional vector to obtain a second combined feature vector; the first combined feature vector, the second combined feature vector and the encoded continuous features are fused to obtain an aggregated feature.

[0137] Each module in the aforementioned click-through rate prediction device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0138] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store click-through rate prediction data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a click-through rate prediction method.

[0139] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0140] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0141] Obtain each coded discrete feature and coded continuous feature of the target user, wherein the coded discrete feature is obtained by feature encoding the discrete features in the user features and the item features, and the coded continuous feature is obtained by feature encoding the continuous features in the user features and the item features; convert each coded discrete feature into a corresponding embedded feature; weight each embedded feature by feature importance to obtain each weighted embedded feature; perform explicit low-order feature crossover on each weighted embedded feature to obtain a low-order combination feature; perform explicit high-order feature crossover on each embedded feature to obtain a high-order combination feature; perform feature aggregation on the low-order combination features, the high-order combination features and the coded continuous features to obtain an aggregated feature, and predict the click rate of the target user based on the aggregated feature.

[0142] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0143] Predicting an attention score corresponding to each of the embedded features, wherein the attention score is used to characterize the importance of the embedded feature; and weighting each of the embedded features according to each of the attention scores to obtain each weighted embedded feature.

[0144] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0145] The feature information in each embedded feature is compressed respectively to obtain each information compression feature value; the feature vector composed of each information compression feature value is fully connected a preset number of times to obtain an attention score vector, wherein the attention score vector includes the attention score of each embedded feature.

[0146] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0147] The embedded feature matrices composed of each of the embedded features are linearly transformed into a query weight matrix, a key weight matrix and a value matrix respectively; the query weight matrix, the key weight matrix and the value matrix are subjected to multi-head attention transformation to obtain an output cross feature matrix; the output cross feature matrix is linearly transformed to obtain a cross feature linear transformation matrix; the cross feature linear transformation matrix is used as a new query weight matrix, and the execution step is returned to: the query weight matrix, the key weight matrix and the value matrix are subjected to multi-head attention transformation to obtain an output cross feature matrix, until the number of feature crosses corresponding to the output cross feature matrix reaches a preset number, and the output cross feature matrix is output as a high-order combined feature.

[0148] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0149] The query weight matrix, the key weight matrix and the value matrix are split respectively to obtain a preset number of sub-matrix sets, wherein the sub-matrix sets include the split query matrix of the query weight matrix, the split key matrix of the key weight matrix and the split value matrix of the value matrix; according to the matrix product between the split query matrix and the split key matrix in each sub-matrix set, the attention weight matrix corresponding to each split value matrix is determined respectively; according to each attention weight matrix, the feature importance of each split value matrix is weighted to obtain each weighted split value matrix; each weighted split value matrix is connected to obtain an output cross feature matrix.

[0150] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0151] The low-order combined features are expanded into a one-dimensional vector to obtain a first combined feature vector; the high-order combined features are expanded into a one-dimensional vector to obtain a second combined feature vector; the first combined feature vector, the second combined feature vector and the encoded continuous features are fused to obtain an aggregated feature.

[0152] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0153] Obtain each coded discrete feature and coded continuous feature of the target user, wherein the coded discrete feature is obtained by feature encoding the discrete features in the user features and the item features, and the coded continuous feature is obtained by feature encoding the continuous features in the user features and the item features; convert each coded discrete feature into a corresponding embedded feature; weight each embedded feature by feature importance to obtain each weighted embedded feature; perform explicit low-order feature crossover on each weighted embedded feature to obtain a low-order combination feature; perform explicit high-order feature crossover on each embedded feature to obtain a high-order combination feature; perform feature aggregation on the low-order combination features, the high-order combination features and the coded continuous features to obtain an aggregated feature, and predict the click rate of the target user based on the aggregated feature.

[0154] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0155] Predicting an attention score corresponding to each of the embedded features, wherein the attention score is used to characterize the importance of the embedded feature; and weighting each of the embedded features according to each of the attention scores to obtain each weighted embedded feature.

[0156] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0157] The feature information in each embedded feature is compressed respectively to obtain each information compression feature value; the feature vector composed of each information compression feature value is fully connected a preset number of times to obtain an attention score vector, wherein the attention score vector includes the attention score of each embedded feature.

[0158] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0159] The embedded feature matrices composed of each of the embedded features are linearly transformed into a query weight matrix, a key weight matrix and a value matrix respectively; the query weight matrix, the key weight matrix and the value matrix are subjected to multi-head attention transformation to obtain an output cross feature matrix; the output cross feature matrix is linearly transformed to obtain a cross feature linear transformation matrix; the cross feature linear transformation matrix is used as a new query weight matrix, and the execution step is returned to: the query weight matrix, the key weight matrix and the value matrix are subjected to multi-head attention transformation to obtain an output cross feature matrix, until the number of feature crosses corresponding to the output cross feature matrix reaches a preset number, and the output cross feature matrix is output as a high-order combined feature.

[0160] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0161] The query weight matrix, the key weight matrix and the value matrix are split respectively to obtain a preset number of sub-matrix sets, wherein the sub-matrix sets include the split query matrix of the query weight matrix, the split key matrix of the key weight matrix and the split value matrix of the value matrix; according to the matrix product between the split query matrix and the split key matrix in each sub-matrix set, the attention weight matrix corresponding to each split value matrix is determined respectively; according to each attention weight matrix, the feature importance of each split value matrix is weighted to obtain each weighted split value matrix; each weighted split value matrix is connected to obtain an output cross feature matrix.

[0162] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0163] The low-order combined features are expanded into a one-dimensional vector to obtain a first combined feature vector; the high-order combined features are expanded into a one-dimensional vector to obtain a second combined feature vector; the first combined feature vector, the second combined feature vector and the encoded continuous features are fused to obtain an aggregated feature.

[0164] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0165] Obtain each coded discrete feature and coded continuous feature of the target user, wherein the coded discrete feature is obtained by feature encoding the discrete features in the user features and the item features, and the coded continuous feature is obtained by feature encoding the continuous features in the user features and the item features; convert each coded discrete feature into a corresponding embedded feature; weight each embedded feature by feature importance to obtain each weighted embedded feature; perform explicit low-order feature crossover on each weighted embedded feature to obtain a low-order combination feature; perform explicit high-order feature crossover on each embedded feature to obtain a high-order combination feature; perform feature aggregation on the low-order combination features, the high-order combination features and the coded continuous features to obtain an aggregated feature, and predict the click rate of the target user based on the aggregated feature.

[0166] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0167] Predicting an attention score corresponding to each of the embedded features, wherein the attention score is used to characterize the importance of the embedded feature; and weighting each of the embedded features according to each of the attention scores to obtain each weighted embedded feature.

[0168] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0169] The feature information in each embedded feature is compressed respectively to obtain each information compression feature value; the feature vector composed of each information compression feature value is fully connected a preset number of times to obtain an attention score vector, wherein the attention score vector includes the attention score of each embedded feature.

[0170] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0171] The embedded feature matrices composed of each of the embedded features are linearly transformed into a query weight matrix, a key weight matrix and a value matrix respectively; the query weight matrix, the key weight matrix and the value matrix are subjected to multi-head attention transformation to obtain an output cross feature matrix; the output cross feature matrix is linearly transformed to obtain a cross feature linear transformation matrix; the cross feature linear transformation matrix is used as a new query weight matrix, and the execution step is returned to: the query weight matrix, the key weight matrix and the value matrix are subjected to multi-head attention transformation to obtain an output cross feature matrix, until the number of feature crosses corresponding to the output cross feature matrix reaches a preset number, and the output cross feature matrix is output as a high-order combined feature.

[0172] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0173] The query weight matrix, the key weight matrix and the value matrix are split respectively to obtain a preset number of sub-matrix sets, wherein the sub-matrix sets include the split query matrix of the query weight matrix, the split key matrix of the key weight matrix and the split value matrix of the value matrix; according to the matrix product between the split query matrix and the split key matrix in each sub-matrix set, the attention weight matrix corresponding to each split value matrix is determined respectively; according to each attention weight matrix, the feature importance of each split value matrix is weighted to obtain each weighted split value matrix; each weighted split value matrix is connected to obtain an output cross feature matrix.

[0174] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0175] The low-order combined features are expanded into a one-dimensional vector to obtain a first combined feature vector; the high-order combined features are expanded into a one-dimensional vector to obtain a second combined feature vector; the first combined feature vector, the second combined feature vector and the encoded continuous features are fused to obtain an aggregated feature.

[0176] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0177] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0178] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0179] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A click rate prediction method, characterized in that: The method comprises: Obtaining each coded discrete feature and coded continuous feature of the target user, wherein the coded discrete features are obtained by feature encoding discrete features in the user features and item features, and the coded continuous features are obtained by feature encoding continuous features in the user features and item features; Converting each of the encoded discrete features into a corresponding embedded feature; Performing feature importance weighting on each of the embedded features to obtain weighted embedded features; Performing explicit low-order feature intersection on each of the weighted embedded features to obtain a low-order combined feature; Performing explicit high-order feature intersection on each of the embedded features to obtain a high-order combined feature; The low-order combined features, the high-order combined features, and the coded continuous features are subjected to feature aggregation to obtain aggregated features, and click-through rate prediction is performed on the target user based on the aggregated features.

2. The method according to claim 1, characterized in that The step of weighting the importance of each embedded feature to obtain each weighted embedded feature includes: Predicting an attention score corresponding to each of the embedded features, wherein the attention score is used to represent the importance of the embedded feature; According to the attention scores, the embedded features are weighted to obtain weighted embedded features.

3. The method according to claim 2, characterized in that The predicting the attention score corresponding to each of the embedded features includes: Compressing the feature information in each embedded feature to obtain a compressed feature value of each information; Perform a preset number of full connections on the feature vectors composed of the information compression feature values to obtain an attention score vector, wherein the attention score vector includes the attention score of each embedded feature.

4. The method according to claim 1, wherein The explicit high-order feature intersection of each of the embedded features to obtain a high-order combined feature includes: Linearly transforming the embedded feature matrix composed of each of the embedded features into a query weight matrix, a key weight matrix and a value matrix respectively; Performing a multi-head attention transformation on the query weight matrix, the key weight matrix, and the value matrix to obtain an output cross-feature matrix; Performing a linear transformation on the output cross-feature matrix to obtain a cross-feature linear transformation matrix; The cross-feature linear transformation matrix is used as a new query weight matrix, and the execution step is returned to: the query weight matrix, the key weight matrix and the value matrix are subjected to multi-head attention transformation to obtain an output cross-feature matrix, until the number of feature crosses corresponding to the output cross-feature matrix reaches a preset number, and the output cross-feature matrix is output as a high-order combined feature.

5. The method according to claim 4, characterized in that The multi-head attention transformation is performed on the query weight matrix, the key weight matrix, and the value matrix to obtain an output cross-feature matrix, including: Splitting the query weight matrix, the key weight matrix, and the value matrix respectively to obtain a preset number of sub-matrix sets, wherein the sub-matrix sets include query matrices split from the query weight matrix, key matrices split from the key weight matrix, and value matrices split from the value matrix; Determining the attention weight matrix corresponding to each of the split value matrices according to the matrix product between the split query matrix and the split key matrix in each of the submatrix sets; According to each of the attention weight matrices, weighting the feature importance of each of the split value matrices to obtain each weighted split value matrix; The weighted split value matrices are connected to obtain an output cross-feature matrix.

6. The method according to claim 1, characterized in that The step of aggregating the low-order combined features, the high-order combined features, and the coded continuous features to obtain aggregated features includes: Expanding the low-order combined features into a one-dimensional vector to obtain a first combined feature vector; Expanding the high-order combined features into a one-dimensional vector to obtain a second combined feature vector; The first combined feature vector, the second combined feature vector, and the encoded continuous feature are fused to obtain an aggregated feature.

7. A click rate prediction device, characterized in that: The device comprises: a feature processing module, configured to obtain coded discrete features and coded continuous features of the target user, wherein the coded discrete features are obtained by feature encoding discrete features among the user features and item features, and the coded continuous features are obtained by feature encoding continuous features among the user features and item features; An embedding feature conversion module, configured to convert each of the encoded discrete features into a corresponding embedding feature; A feature importance weighting module, configured to weight the importance of each embedded feature to obtain weighted embedded features; An explicit low-order feature crossover module, configured to perform explicit low-order feature crossover on each of the weighted embedded features to obtain a low-order combined feature; An explicit high-order feature intersection module, configured to perform explicit high-order feature intersection on each of the embedded features to obtain a high-order combined feature; The prediction module is used to perform feature aggregation on the low-order combined features, the high-order combined features and the coded continuous features to obtain aggregated features, and perform click-through rate prediction on the target user based on the aggregated features.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images