Click rate prediction method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a click rate prediction method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining user attribute information of a target user and object attribute information of a click object, and converting the user attribute information and the object attribute information into multi-domain embedded features; predicting the probability of interaction between the target user and the click object based on each embedded feature through a preset graph neural network; the preset graph neural network comprises a plurality of graph network layers, and the process of performing feature extraction on each graph network layer further comprises the following steps: constructing an initial interaction feature based on the input feature; performing average pooling on the initial interaction features to obtain average pooling features, and performing maximum pooling on the initial interaction features to obtain maximum pooling features; and according to the average pooling feature and the maximum pooling feature, performing feature enhancement on the initial interaction feature to obtain an enhanced interaction feature, and taking the enhanced interaction feature as an input feature of a next graph network layer. By adopting the method, the click rate prediction accuracy can be improved.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a click-through rate prediction method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rapid development of internet information recommendation technology, broad click-through rate (CTR) prediction technology has emerged. This technology aims to estimate the probability of users clicking on recommended content, and its accuracy significantly impacts the conversion rate of recommended content and user experience.

[0003] Traditional techniques often enrich feature representations by modeling feature interactions. Among these, graph neural networks learn node representations by aggregating neighbor information, effectively capturing high-order interaction relationships between features.

[0004] However, because the feature aggregation mechanism of graph neural networks relies on the assumption of similarity between neighboring nodes, some dissimilar but highly predictive feature combinations cannot be effectively captured. This makes it difficult for the model to fully explore beneficial feature interaction patterns in complex scenarios, resulting in low accuracy in click-through rate prediction. Summary of the Invention

[0005] Therefore, it is necessary to provide a click-through rate (CTR) prediction method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of CTR prediction in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a click-through rate prediction method, including:

[0007] Obtain user attribute information of the target user and object attribute information of the clicked object, and convert the user attribute information and object attribute information into multi-domain embedding features;

[0008] By using a pre-defined graph neural network, the probability of a target user interacting with a clicked object is predicted based on each embedded feature;

[0009] The preset graph neural network is trained using historical click data. The preset graph neural network includes multiple graph network layers, and the method further includes the following steps during feature extraction at each graph network layer:

[0010] Construct initial interaction features based on input features;

[0011] Average pooling is performed on the initial interaction features to obtain average pooled features, and max pooling is performed on the initial interaction features to obtain max pooled features.

[0012] Based on the average pooling feature and the max pooling feature, the initial interaction feature is enhanced to obtain the enhanced interaction feature, which is then used as the input feature of the next graph network layer.

[0013] In one embodiment, constructing initial interaction features based on input features includes:

[0014] Each input feature is combined in pairs to form multiple feature pairs. The interaction effect of each feature pair is evaluated to obtain the interaction effect metric of each feature pair.

[0015] Construct an interaction weight matrix based on each interaction effect metric;

[0016] The initial interactive features are obtained by linearly transforming the element-wise product of each feature pair based on the interaction weight matrix.

[0017] In one embodiment, the interaction weight matrix includes the interaction weights of each feature pair; constructing the interaction weight matrix based on each interaction effect metric includes:

[0018] Sort the feature pairs from highest to lowest according to their interaction effect metrics;

[0019] A predetermined number of feature pairs with higher ranking are identified as beneficial interaction feature pairs. Based on the interaction effect metric of the beneficial interaction feature pairs, the interaction weight of the beneficial interaction feature pairs is determined.

[0020] Set the interaction weights of feature pairs other than beneficial interaction feature pairs to a preset minimum value.

[0021] In one embodiment, for any pair of features, the interaction effect of each feature pair is evaluated to obtain an interaction effect metric for each feature pair, including:

[0022] Use the element-wise product of the feature pairs as the initial interaction signal;

[0023] The initial interaction signal is linearly transformed by a preset first weight matrix to obtain a first linear transformation result, and the first linear transformation result is mapped by a preset first nonlinear activation function to obtain intermediate features;

[0024] The intermediate features are linearly transformed by a preset second weight matrix to obtain a second linear transformation result. The second linear transformation result is then mapped by a preset second nonlinear activation function to obtain an interaction effect metric.

[0025] In one embodiment, after obtaining the enhanced interaction features corresponding to each graph network layer, the method further includes:

[0026] The target interaction feature is obtained by average pooling each enhanced interaction feature.

[0027] By performing a nonlinear transformation on the target interaction features, the target interaction features are mapped to the probability of the target user interacting with the clicked object.

[0028] In one embodiment, enhanced interaction features are obtained by performing feature enhancement on the initial interaction features based on average pooling features and max pooling features, including:

[0029] The initial interaction features are calibrated based on the average pooling features to obtain the calibrated interaction features;

[0030] By aggregating the calibrated interaction features and the max-pooling features, we obtain the enhanced interaction features.

[0031] Secondly, this application also provides a click-through rate prediction device, comprising:

[0032] The acquisition module is used to acquire user attribute information of the target user and object attribute information of the clicked object, and convert the user attribute information and object attribute information into multi-domain embedding features.

[0033] The prediction module is used to predict the probability of a target user interacting with a clicked object based on various embedded features using a pre-defined graph neural network.

[0034] The preset graph neural network is trained using historical click data. The preset graph neural network includes multiple graph network layers. During feature extraction in each graph network layer, the prediction module is also used to construct initial interaction features based on the input features. The initial interaction features are averaged by pooling to obtain average pooled features, and max-pooled by max-pooling to obtain max-pooled features. Based on the average pooled features and max-pooled features, the initial interaction features are augmented to obtain augmented interaction features, which are then used as input features for the next graph network layer.

[0035] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0036] Obtain user attribute information of the target user and object attribute information of the clicked object, and convert the user attribute information and object attribute information into multi-domain embedding features;

[0037] By using a pre-defined graph neural network, the probability of a target user interacting with a clicked object is predicted based on each embedded feature;

[0038] The preset graph neural network is trained using historical click data. The preset graph neural network includes multiple graph network layers. During feature extraction at each graph network layer, the processor executes the following steps when running the computer program:

[0039] Construct initial interaction features based on input features;

[0040] Average pooling is performed on the initial interaction features to obtain average pooled features, and max pooling is performed on the initial interaction features to obtain max pooled features.

[0041] Based on the average pooling feature and the max pooling feature, the initial interaction feature is enhanced to obtain the enhanced interaction feature, which is then used as the input feature of the next graph network layer.

[0042] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0043] Obtain user attribute information of the target user and object attribute information of the clicked object, and convert the user attribute information and object attribute information into multi-domain embedding features;

[0044] By using a pre-defined graph neural network, the probability of a target user interacting with a clicked object is predicted based on each embedded feature;

[0045] The preset graph neural network is trained using historical click data. The preset graph neural network includes multiple graph network layers. During feature extraction at each graph network layer, the computer program, when executed by the processor, also performs the following steps:

[0046] Construct initial interaction features based on input features;

[0047] Average pooling is performed on the initial interaction features to obtain average pooled features, and max pooling is performed on the initial interaction features to obtain max pooled features.

[0048] Based on the average pooling feature and the max pooling feature, the initial interaction feature is enhanced to obtain the enhanced interaction feature, which is then used as the input feature of the next graph network layer.

[0049] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0050] Obtain user attribute information of the target user and object attribute information of the clicked object, and convert the user attribute information and object attribute information into multi-domain embedding features;

[0051] By using a pre-defined graph neural network, the probability of a target user interacting with a clicked object is predicted based on each embedded feature;

[0052] The preset graph neural network is trained using historical click data. The preset graph neural network includes multiple graph network layers. During feature extraction at each graph network layer, the computer program, when executed by the processor, also performs the following steps:

[0053] Construct initial interaction features based on input features;

[0054] Average pooling is performed on the initial interaction features to obtain average pooled features, and max pooling is performed on the initial interaction features to obtain max pooled features.

[0055] Based on the average pooling feature and the max pooling feature, the initial interaction feature is enhanced to obtain the enhanced interaction feature, which is then used as the input feature of the next graph network layer.

[0056] The aforementioned click-through rate prediction method, apparatus, computer equipment, computer-readable storage medium, and computer program product first convert user attribute information and click object attribute information into multi-domain embedded features. Then, these multi-domain embedded features are input into a pre-trained graph neural network for interaction probability prediction. By improving the graph neural network structure, it overcomes the technical deficiency of traditional graph neural networks, which suffer from the loss of high-value feature combinations due to reliance on neighbor similarity assumptions. Specifically, in each graph network layer, initial interaction features are first constructed, followed by average pooling and max pooling operations. Average pooling stably extracts universally existing contextual patterns representing the overall distribution background, while max pooling can keenly identify locally sparse but strong interaction signals. Furthermore, the initial interaction features are enhanced by fusing the pooling results with these two different focuses, enabling the graph neural network to simultaneously optimize feature representation based on global patterns and locally prominent signals. This dual-path design allows the graph neural network to move beyond solely relying on the similarity of neighbor node features for information aggregation, simultaneously considering both widely existing conventional associations and dissimilar but highly predictive feature combinations. Therefore, even in complex scenarios, the model can more comprehensively uncover various beneficial feature interaction patterns, thereby significantly improving the accuracy of click-through rate prediction. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart illustrating a click-through rate prediction method in one embodiment of this application;

[0059] Figure 2 This is a flowchart illustrating the feature extraction process implemented by the network layer in one embodiment of this application;

[0060] Figure 3 This is a flowchart illustrating the process of constructing initial interaction features in one embodiment of this application;

[0061] Figure 4 This is a schematic diagram of a graph neural network architecture in another embodiment of this application;

[0062] Figure 5 This is a schematic diagram showing the performance comparison results of graph neural networks in another embodiment of this application;

[0063] Figure 6 This is a structural block diagram of a click-through rate prediction device in one embodiment of this application;

[0064] Figure 7 This is an internal structural diagram of a computer device in one embodiment of this application. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] With the rapid development of internet information recommendation technology, click-through rate (CTR) prediction technology has emerged. This technology aims to estimate the probability of users clicking on recommended content. Its prediction accuracy not only directly affects the conversion rate and commercial value of recommended content in internet companies and e-commerce platforms, but also plays a key role in improving user satisfaction, thus attracting widespread attention from the research community.

[0067] Traditional techniques often improve prediction performance by modeling feature interactions to enrich feature representations. While models like factorization machines can capture low-order feature interactions, they struggle to effectively model complex high-order interactions. In recent years, graph neural networks (GNNs) have been introduced into this field as an effective solution. They learn node embeddings by iteratively aggregating features from neighboring nodes layer by layer, thus explicitly capturing high-order interactions between nodes in the graph. For example, the Fi-GNN (Factorization-Guided Graph Neural Network) method connects each pair of features into a fully connected graph and uses a gated graph neural network to model interactions on the feature graph; GraphFM (Graph Factorization Machine) extends factorization machines with graph neural networks to address their inability to capture high-order interactions.

[0068] However, existing graph neural network-based methods have inherent limitations. On the one hand, their feature interaction mechanisms rely on the assumption that neighboring nodes share similar features, updating node representations by smoothly aggregating neighbor information. This is inconsistent with the essential nature of feature interaction modeling, which requires identifying the intrinsic relationships between different features with predictive value. Consequently, some dissimilar but highly predictive feature combinations are difficult to capture effectively. On the other hand, these methods often treat all neighbor interactions equally during the aggregation process, lacking the ability to distinguish the degree of benefit of interactions. This makes it difficult for the model to fully explore high-quality feature interaction patterns in complex scenarios, thus limiting further improvements in click-through rate prediction accuracy.

[0069] In an exemplary embodiment, a click-through rate prediction method is provided. This embodiment illustrates the application of this method to a terminal, where the terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. It is understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, as... Figure 1 As shown, the method includes steps 102-104. Wherein:

[0070] Step 102: Obtain the user attribute information of the target user and the object attribute information of the clicked object, and convert the user attribute information and object attribute information into multi-domain embedding features.

[0071] The user attribute information can refer to a data set describing user characteristics, including at least one of the following: user's age, gender, historical behavior sequence, and interest tags. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) involved in this embodiment are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with relevant regulations. The acquisition, storage, use, and processing of data in this application's technical solution all comply with relevant national laws and regulations.

[0072] Clickable objects can refer to physical items displayed to users in a digital interface that can be interacted with by users, including at least one of the following: advertisements, products, services, function entry points, and content. Content includes articles in news and information applications, short or long videos on video platforms, songs or playlists in music applications, and posts on social media.

[0073] Object attribute information can refer to a set of data describing the clicked object, including at least one of the following: category, price, text description, image features, etc.

[0074] Multi-domain embedding features refer to converting raw information from different sources or categories into vector representations of a unified dimension. For example, information from different domains, such as a user's age, gender, purchase history, and the text description and image features of the clicked object, can be mapped to dense vectors of a fixed dimension. The division of the embedding feature domains can correspond one-to-one with the types of raw information, or it can be determined according to actual needs. This embodiment does not impose any restrictions on this.

[0075] For example, user attribute information such as basic attributes (e.g., demographic information, user profile information), behavioral attribute information (e.g., historical click sequences, dwell time), and object attribute information such as content category, text features, and visual features of the clicked object can be obtained from a pre-built database or real-time logs. This raw attribute information typically exists in heterogeneous data formats, including numerical, textual, and image types. Through a predefined embedding layer, this raw attribute information can be mapped into fixed-dimensional vectors. Numerical information can be normalized, while textual and image information can be transformed using a pre-trained encoder, ultimately forming a structured set of multi-domain embedded features. These transformed embedded features retain the semantic meaning of the original attribute information and possess mathematical properties suitable for neural network processing.

[0076] Step 104: Using a preset graph neural network, predict the probability of the target user interacting with the clicked object based on each embedded feature.

[0077] Among them, a pre-set graph neural network can refer to a graph neural network model that has been trained using historical data before the model is deployed online, and whose structure and model parameters are fixed.

[0078] For example, after obtaining multi-domain embedding features, these features are fed into a graph neural network pre-trained using a large amount of historical click data. This graph neural network processes these embedding features using complex mapping rules learned from the historical click data and ultimately outputs a probability value between 0 and 1. This probability value represents the likelihood that the graph neural network predicts the target user will interact with the clicked object.

[0079] As an example, the training process of a pre-defined graph neural network may include: acquiring user attribute sample information of sample users, object attribute sample information of sample clicked objects, and actual interaction labels between sample users and sample clicked objects. These actual interaction labels can be binary labels recording whether the user clicked the object; for example, 1 for clicked and 0 for not clicked. The user attribute sample information and object attribute sample information are converted into multi-domain sample embedding features. Using the graph neural network to be trained, training probability predictions of user-clicked object interactions are made based on these sample embedding features. The difference between the training probability predictions and the actual interaction labels is calculated, and the gradient is calculated based on the loss through backpropagation. The model parameters of the graph neural network to be trained are updated based on the gradient. This process is repeated iteratively to optimize the model parameters until the loss converges or the number of iterations reaches a preset threshold, resulting in a trained graph neural network.

[0080] In one exemplary embodiment, the preset graph neural network includes multiple graph network layers, such as... Figure 2 As shown, the method also includes the following steps during feature extraction at each graph network layer:

[0081] Step 202: Construct initial interaction features based on input features.

[0082] In a graph neural network (GNN), the graph network layer is the most basic data processing unit, primarily responsible for performing feature information transfer and aggregation operations on graph-structured data. In a typical GNN, multiple layers are stacked sequentially to form a hierarchical information processing system. Each layer focuses on processing node relationships of a specific order; for example, the first layer captures the feature interactions of direct neighbors, the second layer captures relationships within two hops, and so on. This hierarchical design allows GNNs to gradually uncover complex high-order association patterns in data through the collaborative work of multiple layers. Functionally, the graph network layer performs operations such as neighbor aggregation, nonlinear transformation, and representation updates on node features, enabling the model to simultaneously integrate the node's own attribute information and the topological relationships within the graph structure, ultimately generating node representations that contain both rich semantics and structural information.

[0083] Initial interaction features can refer to feature representations that represent the potential interaction relationships between features, obtained by combining or transforming input features through specific mathematical operations. Initial interaction features contain the original signals of interaction between features.

[0084] For example, within each layer of the predefined graph neural network, for the first layer, the model input, i.e., the multi-domain embedding features, can be used as the input features; for other layers, the output features of the previous layer can be used as the input features. Then, through specific operations, a new feature representation that can initially express the pairwise interaction relationships between features is generated, i.e., the initial interaction features.

[0085] As an example, the characteristic operation for generating initial interactive features may include treating the input features as nodes in a fully connected graph and processing all node features through a learnable linear transformation layer.

[0086] As another example, the characteristic operation for generating initial interactive features may include: directly calculating the element-wise product between each input feature.

[0087] Step 204: Perform average pooling on the initial interaction features to obtain average pooled features, and perform max pooling on the initial interaction features to obtain max pooled features.

[0088] For example, after generating the initial interaction features, average pooling and max pooling operations can be performed on the generated initial interaction features. Average pooling calculates the average value of the initial interaction features along a specified dimension (e.g., the sequence length dimension), resulting in an average pooled feature that contains overall distribution characteristics and contextual information. Max pooling extracts the maximum value of the initial interaction features along the same dimension, resulting in a max pooled feature containing the most significant and strongest interaction signal. For example, if the initial interaction features are a tensor of shape [n, d], where n represents the number of features and d represents the feature dimension, average pooling calculates the average value of the n features along each dimension, outputting a feature vector of shape [1, d]; max pooling selects the maximum value along each dimension, also outputting a feature vector of shape [1, d]. These two pooling operations extract key information from the initial interaction features from different perspectives.

[0089] Step 206: Based on the average pooling features and max pooling features, perform feature enhancement on the initial interaction features to obtain enhanced interaction features, and use the enhanced interaction features as the input features of the next graph network layer.

[0090] For example, an enhanced interaction feature can be generated by fusing average pooling features, max pooling features, and initial interaction features using a predefined fusion function. This enhanced interaction feature retains the original interaction information while incorporating global statistical properties and significant local features, enabling a more comprehensive representation of the interaction relationships between features. This enhanced interaction feature will be used as the output of the current graph network layer and passed to the next graph network layer as input for a new round of feature extraction, thereby achieving a progressively deeper mining and refinement of feature interaction information.

[0091] As an example, the way to fuse average pooling features, max pooling features, and initial interaction features can include: using average pooling features as gating weights to perform dot product with the initial interaction features, and then performing matrix multiplication or concatenation with the max pooling features.

[0092] As another example, the way to fuse average pooling features, max pooling features, and initial interaction features can be: concatenating the average pooling features and max pooling features to form a comprehensive context vector, and then mapping it to a space with the same dimension as the initial interaction features through a learnable transformation (such as a fully connected layer); subsequently, adding or concatenating the transformed context vector with the initial interaction features element-wise to generate enhanced interaction features.

[0093] The aforementioned click-through rate (CTR) prediction method first converts user attribute information and click object attribute information into multi-domain embedded features. These features are then input into a pre-trained graph neural network (Graph Neural Network) for interaction probability prediction. By improving the Graph Neural Network structure, it overcomes the technical deficiency of traditional Graph Neural Networks, which rely on the assumption of neighbor similarity and thus lose high-value feature combinations. Specifically, in each Graph Neural Network layer, initial interaction features are first constructed, followed by average pooling and max pooling operations. Average pooling stably extracts universally existing contextual patterns representing the overall distribution background, while max pooling can keenly identify locally sparse but strong interaction signals. Furthermore, the initial interaction features are enhanced by fusing the pooling results with these two different focuses, enabling the Graph Neural Network to optimize feature representation based on both global patterns and locally prominent signals. This dual-path design allows the Graph Neural Network to move beyond solely relying on the similarity of neighbor node features for information aggregation, simultaneously considering both widely existing conventional associations and dissimilar but highly predictive feature combinations. Therefore, even in complex scenarios, the model can more comprehensively uncover various beneficial feature interaction patterns, significantly improving the accuracy of CTR prediction.

[0094] In one exemplary embodiment, such as Figure 3 As shown, constructing initial interaction features based on input features includes steps 302 to 306. Wherein:

[0095] Step 302: Combine each input feature in pairs to form multiple feature pairs, evaluate the interaction effect of each feature pair, and obtain the interaction effect metric value of each feature pair.

[0096] It should be noted that traditional graph neural networks, when constructing feature interactions, typically treat all feature combinations equally or aggregate them solely based on feature similarity, failing to distinguish the differences in interaction value between different feature pairs. This indiscriminate treatment makes it difficult for the model to identify and strengthen feature combinations that, although the features are dissimilar, have high value for the prediction task. Simultaneously, it cannot filter out invalid feature interactions that may introduce noise, ultimately limiting the expressive power and accuracy of graph neural networks used for click-through rate prediction.

[0097] In this context, a feature pair refers to a pair consisting of two different input features, and it is the basic unit for interactive analysis.

[0098] Interaction performance evaluation can refer to the process of quantitatively evaluating the contribution of features to click-through rate.

[0099] Interaction performance metrics are used to characterize the strength of the impact of the combination of two features in a feature pair on the click-through rate.

[0100] For example, all input features are iterated through and paired to form multiple feature pairs. For each feature pair, its two corresponding feature vectors are input into a predefined evaluation function, which, through forward computation, outputs a scalar value, namely the interaction measure of the feature pair. This value quantitatively characterizes the importance or strength of the interaction between the two features.

[0101] As an example, the evaluation function could be a function used to calculate the correlation coefficient between feature co-occurrence frequency and click-through rate;

[0102] As another example, the evaluation function can be a parameter-learnable function integrated into a graph neural network. During the training of the graph neural network, the parameters of this evaluation function are updated and optimized along with other network parameters using the gradient descent algorithm.

[0103] Step 304: Construct an interaction weight matrix based on each interaction effect metric.

[0104] The interaction weight matrix is ​​a matrix composed of the interaction effect measures of each feature pair. The value of each element represents the interaction effect measure between the two input features in the corresponding feature pair. As an example, the interaction weight matrix can be an N×N matrix, where N is the total number of features. The element W(i,j) in this matrix represents the interaction weight between the i-th feature and the j-th feature. The interaction weight between the i-th feature and the j-th feature can be equal to the interaction effect measure between the i-th feature and the j-th feature.

[0105] For example, first, an empty matrix with the number of rows and columns equal to the number of input features is constructed or obtained. Then, for a feature pair consisting of input feature i and input feature j, the corresponding interaction weight is determined based on their respective interaction effect metrics. The positions in the empty matrix corresponding to input features i and j are then assigned the interaction weights of the feature pair consisting of these two input features. This results in an interaction weight matrix, which provides a global view that clearly shows the interaction weights between any two input features. The values ​​are determined by the importance of their interaction, rather than by pre-defined fixed values.

[0106] The methods for determining interaction weights based on interaction effect metrics include directly using the interaction effect metrics as interaction weights, or mapping the interaction effect metrics to corresponding interaction weights according to a predefined mapping relationship.

[0107] Step 306: Perform a linear transformation on the element-wise product of each feature pair based on the interaction weight matrix to obtain the initial interaction features.

[0108] For example, after obtaining the interaction weight matrix, the element-wise product of the two input features in each feature pair is first calculated to generate the basic interaction signal. Subsequently, the interaction weight matrix is ​​used to perform a weighted linear transformation on the element-wise product results of all feature pairs, and the transformed result can be used as the initial interaction feature.

[0109] As an example, this weighted linear transformation can be achieved by left-multiplying or right-multiplying the interactive weight matrix by a matrix consisting of the product of all its elements.

[0110] In this embodiment, by constructing an interaction weight matrix, the model can proactively evaluate and assign differentiated interaction weights to different feature pairs, thereby selectively enhancing beneficial interactions and suppressing ineffective ones. Based on this, a linear transformation is performed using this interaction weight matrix, allowing the generated initial interaction features to incorporate prior knowledge about the importance of interactions. This mechanism enables the graph neural network layer to learn and infer on a higher-quality, more informative feature base, focusing on beneficial interactions and significantly improving the model's ability to capture key user intent, ultimately enhancing the accuracy of click-through rate prediction.

[0111] In an exemplary embodiment, the interaction weight matrix includes the interaction weights of each feature pair; constructing the interaction weight matrix based on each interaction effect metric includes:

[0112] Sort each feature pair from highest to lowest according to the interaction effect metric; identify a preset number of feature pairs with higher ranking as beneficial interaction feature pairs, and determine the interaction weight of the beneficial interaction feature pairs based on their interaction effect metric; set the interaction weight of other feature pairs except for the beneficial interaction feature pairs to a preset minimum value.

[0113] It's important to note that in traditional graph neural networks, feature maps are typically predefined as fully connected or similar forms, treating all connections between feature pairs equally. This leads to the fact that during information aggregation, information about feature interactions, whether beneficial or not, is passed on and learned together. This makes the model susceptible to noisy interactions and makes it difficult to focus on truly important beneficial interactions, thus limiting the accuracy of click-through rate prediction.

[0114] The preset minimum value is a pre-defined, infinitely close to zero positive number or zero itself, used to effectively shield or greatly weaken the influence of non-beneficial interaction feature pairs in numerical calculations.

[0115] For example, after determining the interaction effect metrics for all feature pairs, all feature pairs are arranged in descending order based on the magnitude of these interaction effect metrics. Then, according to a preset number, such as the top P... K One or the top 100% of P K From the sorted list of feature pairs, the highest-ranking feature pairs are selected and identified as beneficial interaction feature pairs. Then, based on the interaction effect metric of these beneficial interaction feature pairs, their corresponding interaction weights are determined. For all other feature pairs not identified as beneficial interaction feature pairs, their corresponding interaction weights are uniformly set to a preset minimum value to reduce the contribution of these non-beneficial interaction feature pairs to the click-through rate.

[0116] In this embodiment, through global sorting and optimal selection, the graph neural network can actively identify and focus on some beneficial feature interactions while filtering out a massive number of weak or invalid feature interactions. By concentrating model computational resources on learning the most predictive interaction patterns, noise interference is significantly reduced, thereby improving the model's feature representation quality and generalization ability, and increasing the accuracy of click-through rate prediction.

[0117] In an exemplary embodiment, for any pair of features, the interaction effect of each feature pair is evaluated to obtain an interaction effect metric for each feature pair, including:

[0118] The element-wise product of the feature pairs is used as the initial interaction signal; the initial interaction signal is linearly transformed by a preset first weight matrix to obtain the first linear transformation result; the first linear transformation result is mapped by a preset first nonlinear activation function to obtain intermediate features; the intermediate features are linearly transformed by a preset second weight matrix to obtain the second linear transformation result; the second linear transformation result is mapped by a preset second nonlinear activation function to obtain the interaction effect metric.

[0119] It should be noted that relying on a fixed linear calculation pattern for interaction effect evaluation makes it difficult to fully capture the complex nonlinear dependencies between features, thus limiting further improvement in model performance.

[0120] The preset first weight matrix can refer to a predefined learnable parameter matrix used to perform the first linear transformation on the input data. The structure and role of the preset first weight matrix are predefined during the model design phase, but its specific parameter values ​​are not pre-set. Instead, they are determined through backpropagation iterative optimization as part of the graph neural network during model training.

[0121] The preset first nonlinear activation function can refer to a pre-defined nonlinear mapping function used to introduce the nonlinear expressive power of the model, such as ReLU (Rectified Linear Unit), Sigmoid (S-shaped function), Tanh (Hyperbolic Tangent function), etc.

[0122] The predefined second weight matrix can refer to a predefined learnable parameter matrix used to perform a second linear transformation on the input data. The structure and role of the predefined second weight matrix are predefined during the model design phase, but its specific parameter values ​​are not pre-set. Instead, they are determined through backpropagation iterative optimization as part of the graph neural network during model training.

[0123] The preset second nonlinear activation function can refer to a pre-defined nonlinear mapping function used to constrain the output to a specific range. For example, the Sigmoid function can be used to constrain the output to the range (0,1).

[0124] For example, each feature pair needs to undergo interaction effect evaluation. For any feature pair, firstly, the two input features are multiplied element-wise to generate an initial interaction signal with the same dimension as the input features. Then, a first linear transformation is performed on the initial interaction signal using a preset first weight matrix. The result of the first linear transformation is then input into a preset first nonlinear activation function for nonlinear mapping, outputting intermediate features, thus realizing the transformation from the original interaction signal to higher-order features. Next, a second linear transformation is performed on the intermediate features using a preset second weight matrix. The result of the second linear transformation is then input into a preset second nonlinear activation function for nonlinear mapping, outputting a scalar value as an interaction effect metric. This interaction effect metric can characterize the interaction strength evaluation result of the current feature pair.

[0125] In some feasible implementations, the input feature e i With input feature e j Interaction effect metric φ(e) i ,e j This can be represented as:

[0126]

[0127] in, A second nonlinear activation function is preset; The first nonlinear activation function is preset; All of these are learnable parameters.

[0128] In this embodiment, an evaluation network with deep nonlinear expressive power is constructed by combining two consecutive linear transformations with nonlinear activation. This evaluation network can automatically learn and identify complex interaction patterns between features, achieving accurate quantification of interaction strength, thereby significantly improving the model's efficiency in utilizing effective feature interactions and enhancing the accuracy of click-through rate prediction.

[0129] In an exemplary embodiment, after obtaining the enhanced interaction features corresponding to each graph network layer, the method further includes:

[0130] Average pooling is performed on each enhanced interaction feature to obtain the target interaction feature; by performing a non-linear transformation on the target interaction feature, the target interaction feature is mapped to the probability of the target user interacting with the clicked object.

[0131] It's important to note that in graph neural networks, the feature representations learned by different network layers are at different levels of abstraction. Relying solely on the output of the last layer for prediction will ignore potentially useful feature information in shallower layers, thus limiting the full expression of model performance.

[0132] For example, after all graph network layers have completed information transmission, the enhanced interaction features output by all graph network layers are collected. These features are then subjected to average pooling to generate the target interaction feature. This target interaction feature integrates multi-level feature interaction information from low to high order. Subsequently, the target interaction feature can be input into a multilayer perceptron composed of one or more fully connected layers. A nonlinear transformation is performed through a combination of linear transformation and nonlinear activation functions. Finally, the output is mapped to a probability value between 0 and 1 through a sigmoid activation function. This probability value is the predicted probability that the target user will interact with the clicked object.

[0133] In this embodiment, by fusing the enhanced interaction features from each layer using average pooling, the graph neural network can simultaneously utilize both low-order simple interaction patterns and high-order complex interaction relationships to form a more comprehensive and robust feature representation. This allows for the full extraction of beneficial information from all layers of the network, resulting in more accurate interaction probability predictions.

[0134] In an exemplary embodiment, enhanced interaction features are obtained by performing feature enhancement on the initial interaction features based on average pooling features and max pooling features, including:

[0135] The initial interaction features are calibrated based on the average pooling features to obtain calibrated interaction features; the calibrated interaction features and the max pooling features are aggregated to obtain enhanced interaction features.

[0136] It should be noted that using a single aggregation strategy, such as single average pooling or single max pooling, makes it difficult to capture both global contextual information and local salient features simultaneously. This limits the model's ability to express complex interaction patterns, resulting in the loss or weakening of some key information during transmission.

[0137] For example, average pooling features are used as calibration weights and multiplied element-wise with the initial interaction features. Since average pooling features provide global contextual information, element-wise multiplication allows for a readjustment of the importance of each dimension in the initial interaction features, highlighting features consistent with the global context and suppressing inconsistent features, thus obtaining calibrated interaction features. Subsequently, matrix multiplication aggregates the calibrated interaction features with max pooling features, effectively fusing the globally context-calibrated calibrated interaction features with locally salient features to generate enhanced interaction features containing multi-scale information.

[0138] In some feasible implementations, the max pooling feature f max It can be represented as:

[0139]

[0140] Average pooling feature f mean It can be represented as:

[0141]

[0142] in, This indicates max pooling; Indicates average pooling; Represents the learnable weight variables; Indicates the activation function; e represents the input feature e of the k-th layer of the graph network. i With input feature e j Interaction weights.

[0143] The enhanced interactive feature E output by the k-th layer of the graph network k+1 It can be represented as:

[0144]

[0145] in, Represents matrix multiplication. This indicates element-wise multiplication.

[0146] In this embodiment, global context calibration is first performed using average pooling features, and then aggregated with max pooling features, achieving an organic combination of global information and local salient features. This maintains global consistency in feature interactions while highlighting important local features, thereby generating more discriminative feature representations, significantly improving the model's ability to model complex interaction patterns, and enhancing the accuracy of click-through rate prediction.

[0147] In an exemplary embodiment, the training process of a pre-defined graph neural network may include: organizing ad click-through rate data; randomly selecting 70% of the organized dataset for use as a training set, 20% for use as a validation set, and 10% for use as a test set; building a model according to the designed overall architecture diagram of the graph neural network; inputting the training dataset and validation dataset into the model in batches to train the model until the model performance meets the requirements; inputting the test dataset into the pre-trained model and obtaining the prediction output for each test record.

[0148] The overall architecture diagram of the graph neural network is as follows: Figure 4 As shown, e i k E represents the embedding feature of the i-th domain in the k-th layer. k+1 This represents the enhanced interactive features output by the k-th layer of the graph network, and can also be used as the input features of the (k+1)-th layer of the graph network.

[0149] To verify the actual performance of the algorithm proposed in this embodiment, the industry-recognized metrics Logloss (Logarithmic Loss) and AUC (Area Under the Curve) were used for evaluation. The Logloss metric measures the accuracy of the model's predicted probabilities; the closer its value is to zero, the better the model's performance. The AUC metric reflects the model's ability to distinguish between different classes; the closer its value is to 1, the better the model's classification performance. We compared the prediction performance of the algorithm proposed in this study with that of existing mainstream algorithms, such as... Figure 5 The results shown are from Figure 5 It can be seen that the algorithm in this embodiment outperforms the benchmark algorithm in terms of performance.

[0150] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0151] Based on the same inventive concept, this application also provides a click-through rate (CTR) prediction device for implementing the CTR prediction method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more CTR prediction device embodiments provided below can be found in the limitations of the CTR prediction method described above, and will not be repeated here.

[0152] In one exemplary embodiment, such as Figure 6 As shown, a click-through rate prediction device is provided, including: an acquisition module 602 and a prediction module 604, wherein:

[0153] The acquisition module 602 is used to acquire user attribute information of the target user and object attribute information of the clicked object, and convert the user attribute information and object attribute information into multi-domain embedding features.

[0154] The prediction module 604 is used to predict the probability of a target user interacting with a clicked object based on each embedded feature using a preset graph neural network.

[0155] The preset graph neural network is trained using historical click data. The preset graph neural network includes multiple graph network layers. During feature extraction in each graph network layer, the prediction module 604 is also used to construct initial interaction features based on input features. The initial interaction features are averaged pooled to obtain average pooled features, and the initial interaction features are max pooled to obtain max pooled features. Based on the average pooled features and max pooled features, the initial interaction features are enhanced to obtain enhanced interaction features, which are then used as input features for the next graph network layer.

[0156] In one exemplary embodiment, the prediction module 604 is further configured to:

[0157] Each input feature is combined in pairs to form multiple feature pairs. The interaction effect of each feature pair is evaluated to obtain the interaction effect metric of each feature pair.

[0158] Construct an interaction weight matrix based on each interaction effect metric;

[0159] The initial interactive features are obtained by linearly transforming the element-wise product of each feature pair based on the interaction weight matrix.

[0160] In an exemplary embodiment, the interaction weight matrix includes the interaction weights of each feature pair; the prediction module 604 is further configured to:

[0161] Sort the feature pairs from highest to lowest according to their interaction effect metrics;

[0162] A predetermined number of feature pairs with higher ranking are identified as beneficial interaction feature pairs. Based on the interaction effect metric of the beneficial interaction feature pairs, the interaction weight of the beneficial interaction feature pairs is determined.

[0163] Set the interaction weights of feature pairs other than beneficial interaction feature pairs to a preset minimum value.

[0164] In an exemplary embodiment, for any pair of features, the prediction module 604 is further configured to:

[0165] Use the element-wise product of the feature pairs as the initial interaction signal;

[0166] The initial interaction signal is linearly transformed by a preset first weight matrix to obtain a first linear transformation result, and the first linear transformation result is mapped by a preset first nonlinear activation function to obtain intermediate features;

[0167] The intermediate features are linearly transformed by a preset second weight matrix to obtain a second linear transformation result. The second linear transformation result is then mapped by a preset second nonlinear activation function to obtain an interaction effect metric.

[0168] In an exemplary embodiment, after obtaining the enhanced interaction features corresponding to each graph network layer, the prediction module 604 is further configured to:

[0169] The target interaction feature is obtained by average pooling each enhanced interaction feature.

[0170] By performing a nonlinear transformation on the target interaction features, the target interaction features are mapped to the probability of the target user interacting with the clicked object.

[0171] In one exemplary embodiment, the prediction module 604 is further configured to:

[0172] The initial interaction features are calibrated based on the average pooling features to obtain the calibrated interaction features;

[0173] By aggregating the calibrated interaction features and the max-pooling features, we obtain the enhanced interaction features.

[0174] The various modules in the aforementioned click-through rate prediction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0175] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a click-through rate prediction method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0176] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0177] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0178] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0179] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0180] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0181] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0182] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0183] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A click-through rate prediction method, characterized in that, The method includes: Obtain user attribute information of the target user and object attribute information of the clicked object, and convert the user attribute information and the object attribute information into multi-domain embedding features; Using a preset graph neural network, the probability of the target user interacting with the clicked object is predicted based on the embedded features. The preset graph neural network is trained using historical click data; the preset graph neural network includes multiple graph network layers, and the method further includes the following during feature extraction in each graph network layer: Construct initial interaction features based on input features; The initial interaction features are subjected to average pooling to obtain average pooled features, and the initial interaction features are subjected to max pooling to obtain max pooled features. Based on the average pooling feature and the max pooling feature, the initial interaction feature is enhanced to obtain the enhanced interaction feature, which is then used as the input feature of the next graph network layer.

2. The method according to claim 1, characterized in that, The construction of initial interaction features based on input features includes: Each input feature is combined in pairs to form multiple feature pairs. The interaction effect of each feature pair is evaluated to obtain the interaction effect metric of each feature pair. Construct an interaction weight matrix based on the interaction effect metrics described above; Based on the interaction weight matrix, a linear transformation is performed on the element-wise product of each feature pair to obtain the initial interaction features.

3. The method according to claim 2, characterized in that, The interaction weight matrix includes the interaction weights of each of the feature pairs; The step of constructing an interaction weight matrix based on each of the interaction effect metrics includes: The feature pairs are sorted from high to low according to their interaction effect metrics; A predetermined number of feature pairs with higher ranking are identified as beneficial interaction feature pairs, and the interaction weight of the beneficial interaction feature pairs is determined based on the interaction effect metric of the beneficial interaction feature pairs. Set the interaction weights of feature pairs other than beneficial interaction feature pairs to a preset minimum value.

4. The method according to claim 2, characterized in that, For any pair of features, the interaction effect evaluation of each pair of features is performed to obtain an interaction effect metric for each pair of features, including: The element-wise product of the feature pairs is used as the initial interaction signal; The initial interaction signal is linearly transformed by a preset first weight matrix to obtain a first linear transformation result, and the first linear transformation result is mapped by a preset first nonlinear activation function to obtain intermediate features; The intermediate features are linearly transformed by a preset second weight matrix to obtain a second linear transformation result, and the interaction effect metric is obtained by mapping the second linear transformation result to a preset second nonlinear activation function.

5. The method according to any one of claims 1 to 4, characterized in that, After obtaining the enhanced interaction features corresponding to each of the graph network layers, the method further includes: The enhanced interaction features are averaged and pooled to obtain the target interaction features. By performing a nonlinear transformation on the target interaction features, the target interaction features are mapped to the probability that the target user will interact with the clicked object.

6. The method according to any one of claims 1 to 4, characterized in that, The step of enhancing the initial interaction features to obtain enhanced interaction features based on the average pooling features and the max pooling features includes: The initial interaction features are calibrated based on the average pooling features to obtain calibrated interaction features; The calibration interaction features and the max pooling features are aggregated to obtain the enhanced interaction features.

7. A click-through rate prediction device, characterized in that, The device includes: The acquisition module is used to acquire user attribute information of the target user and object attribute information of the clicked object, and convert the user attribute information and the object attribute information into multi-domain embedding features. The prediction module is used to predict the probability that the target user will interact with the clicked object based on the embedded features using a preset graph neural network. The preset graph neural network is trained using historical click data. The preset graph neural network includes multiple graph network layers. During feature extraction in each graph network layer, the prediction module is also used to construct initial interaction features based on input features. The initial interaction features are then subjected to average pooling to obtain average pooled features, and max pooling to obtain max pooled features. Based on the average pooled features and the max pooled features, the initial interaction features are augmented to obtain augmented interaction features, which are then used as input features for the next graph network layer.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images