Click rate prediction method based on multiple interests of user
By designing a click-through rate prediction method based on users' multiple interests, using multi-head self-attention mechanism and feed-forward network to extract users' multiple interests, and combining target items, user profiles and contexts for prediction, it solves the problem that existing models ignore collaborative users. and context field issues, improving the accuracy of click-through rate prediction.
Patent Information
- Application Number
- CN202411362922.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2025-01-21
AI Technical Summary
The existing click-through rate prediction model ignores the interests of associated collaborative users and the target project context fields, resulting in low prediction accuracy.
Designed a clicks prediction method based on the user's multiple interests. By obtaining the data of the target users, collaborative users and target items, it uses a multi -headed self -attention mechanism and feedback network to extract users. Combine the configuration file and context, and enter MLP for click -through rate prediction.
By associating the collaborative user and target project context fields, the accuracy of click-through rate prediction is improved, and the accuracy of predicting the target user's click-through rate on the target project is improved.
Smart Images
Figure CN119336988A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of click-through rate prediction (CTR) in a recommendation system, and in particular to a click-through rate prediction method based on user's multiple interests. Background Art
[0002] Click-through rate (CTR) prediction is widely used to tailor product recommendations for users, which can significantly increase overall revenue. Therefore, effectively designing CTR prediction models has become a focus of academic and industrial attention.
[0003] In the prior art, multi-interest modeling is often used to predict click-through rate. For example, GRU and attention mechanisms are used to simulate interest representation and its evolution. A memory-based architecture is introduced to capture user interests from a large number of behavioral data sequences. Alternatively, multiple historical sessions in a user behavior sequence are used, and Bi-LSTM is applied to simulate how user interests evolve and interact between sessions. In addition to deep sequence models, technologies such as dynamic routing, capsule networks, and attention mechanisms are also used in the prior art to obtain multiple interest representations for users. Dynamic routing technology is used to achieve multi-interest expression of users, and capsule networks are introduced for multi-interest extraction. However, existing methods mainly consider multiple interests of target users from a single perspective, ignoring the user-user correlation between the target user and collaborative users with similar interests; and ignoring the impact of other target item fields and other fields in the user and context on interest modeling.
[0004] Therefore, a CTR prediction method is needed that can associate collaborative users and context fields of target items. Summary of the invention
[0005] In view of this, the present invention provides a click rate prediction method based on user multiple interests to solve the problem that the click rate prediction model ignores the interests of associated collaborative users and the context fields of the target project.
[0006] To this end, the present invention provides the following technical solutions:
[0007] A click rate prediction method based on user's multiple interests, comprising:
[0008] Obtain target user profiles, target user behaviors, target projects, target project contexts, and collaborative user behaviors;
[0009] Build a click-through rate prediction model based on user's multiple interests;
[0010] The acquired data is converted into a low-dimensional dense vector through the data embedding layer;
[0011] The model includes a target user multi-interest extraction module, a collaborative user multi-interest extraction module, and a click rate prediction module;
[0012] The target user multi-interest extraction module uses the target user behavior vector to obtain a refined target user behavior sequence representation through a multi-head self-attention mechanism; models the interaction of each field pair between the behavior field and the query field to obtain the target user's interests;
[0013] The collaborative user multi-interest extraction module uses the collaborative user behavior vector to obtain the multi-interest of each collaborative user through a multi-head self-attention machine and a feedforward network; aggregates the multi-interest of each collaborative user to obtain the interest of each collaborative user;
[0014] The click rate prediction module connects the vector representations of the target user's multiple interests, each collaborative user's interests, the target project, the user profile, and the context; inputs the connected vector into the MLP to predict the target user's click probability on the target project.
[0015] Furthermore, the step of converting the acquired data into a low-dimensional dense vector through a data embedding layer includes:
[0016] Each feature is encoded into a one-hot vector via a high-dimensional sparse binary encoding and converted to a low-dimensional dense vector:
[0017] Target user behavior V u , Collaborative User Behavior m , user profile A, target item I and target item context C, the corresponding embedding vectors are X u , X m 、e a 、e i and e c ;
[0018] is the embedding vector of the target user’s t-th behavior;
[0019] Yes F u The embedding vector of all behaviors of the kth collaborative user in ;
[0020] Among them, the collaborative user behavior of user u can be defined as is the user behavior of the t-th collaborative user;
[0021] Represents the collection of all feature fields from the target project, user profile, and target project context aspects.
[0022] Furthermore, the method of using the target user behavior vector to obtain a refined target user behavior sequence representation through a multi-head self-attention mechanism includes:
[0023] Set the number of heads to N, and connect the multi-head calculation results to form a detailed representation of each user behavior sequence:
[0024]
[0025] in, is the u-th behavior embedding; and is the weight matrix of the u-th head; Q, K, and V are query, key, and value respectively; d u It is the dimension of query, key, and value.
[0026] Furthermore, the interaction between each field pair between the behavior field and the query field is modeled to obtain the interests of the target user, including:
[0027]
[0028] in, is the u-th user behavior embedding; Indicates weight; Interest u Indicates the multiple interests of the target user.
[0029] Furthermore, the method of using the collaborative user behavior vector to obtain multiple interests of each collaborative user through a multi-head self-attention machine and a feedforward network includes:
[0030] The multi-head attention layer extracts the relationship between each collaborative user's items and learns to refine the user interest representation:
[0031]
[0032] in is u m User behavior embedding; and is the weight matrix of the u-th head; Q, K, and V are query, key, and value respectively; d u are the dimensions of query, key, and value;
[0033] Connect the multi-head computation results and input them into the feed-forward network to obtain the interest representation of each collaborative user:
[0034] r n =FFN(concat(head1,head2,…,head N )W O )
[0035] Where N is the number of heads, FFN(·) is the feedforward network, and W O is a linear matrix.
[0036] Furthermore, the aggregating multiple interests of each collaborative user to obtain each collaborative user interest includes:
[0037]
[0038] in, b a is the learning parameter; a n represents the similarity weight between the target user and the collaborative user; L i is the item behavior length of target item i.
[0039] Furthermore, the concatenated vector is input into the MLP to predict the click probability of the target user on the target item:
[0040] imput=concat(Interest u ,Interest r ,e a ,e i ,e c )
[0041]
[0042] Among them, e a 、e i and e c are the embedding vectors of user profile A, target item I, and target item context C.
[0043] Furthermore, the click rate prediction loss function based on user's multiple interests is:
[0044]
[0045] Where N is the number of training samples; Θ is all trainable parameters; y i ∈{0,1} represents the label; λ||Θ||2 represents the L2 regularization term.
[0046] Advantages and positive effects of the present invention:
[0047] The present invention designs a click rate prediction method based on user multi-interest from the perspective of multi-interest to accurately model the interest representation of target users and collaborative users. The present invention not only focuses on different interests in user historical behavior data, but also captures similar interests between users. In addition, all project / user / context fields are used as queries in the focus unit to model the target user's interests, so that better interest representations can be learned. This improves the accuracy of predicting the click rate of the target user on the target project. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0049] Figure 1 is a flow chart of an embodiment of the present invention;
[0050] Figure 2 This is a schematic diagram of an embodiment of the present invention. DETAILED DESCRIPTION
[0051] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0052] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0053] The present invention implements a new deep user multi-interest modeling method and proposes a two-pronged approach to extract different interests to predict click-through rate. On the one hand, the target user multi-interest extraction network and the target item / user / context fields are used as queries within the attention unit to effectively model and extract the target user's multiple interests, so as to solve the problem that the prior art ignores the influence of other target item fields and other fields in the user and context on interest modeling. On the other hand, the collaborative user multi-interest extraction network adopts a multi-head self-attention mechanism to derive multi-interest mappings for collaborative users, so as to solve the problem that the prior art only considers the target user's multiple interests from a single perspective, ignoring the user-user correlation between the target user and collaborative users with similar interests.
[0054] Combination Figure 1 , 2 , the method of the present invention is further described:
[0055] S1. Data preprocessing:
[0056] The input data includes: user profiles, user behaviors, target items, contexts, and collaborative user behaviors. There are interactions between users and items (such as clicks or ratings), which are recorded in historical data.
[0057] Based on historical data, the behavior sequence of target users and collaborative users is obtained:
[0058] User behavior: Given user u, user behavior B u is a time-ordered list of items that user u interacted with, i.e., V u =[v1,v2,…,v N ], where v t is the tth interactive item, N is B u Length.
[0059] Collaborative user behavior: Get the users who have interacted with the target project and their behaviors. Each user has a list of collaborative users who interacted with the target project, defined as U m =[u1,u2,…,u L ], where u t is the t-th collaborative user, and L is the number of collaborative users of this user. Each collaborative user also has a user behavior. Therefore, the collaborative user behavior of user u can be defined as in is the user behavior of the t-th collaborative user.
[0060] Therefore, the CTR prediction task is defined as: given a user behavior V u , Collaborative user behavior u , user profile A, target item I and context C, predict the probability that user u clicks on target item l.
[0061] S2. Data Embedding:
[0062] Through the feature encoding method One-hot, the embedding layer converts the input features into an embedding vector.
[0063] S21. Encode each feature into a one-hot vector using high-dimensional sparse binary coding.
[0064] S22, one-hot vectors are converted into low-dimensional dense vectors through the embedding layer.
[0065] User behavior V u , Collaborative User Behaviorm , user profile A, target item I and context C are obtained through the embedding layer to obtain the embedding vector of each input respectively u , X m 、e a 、e i 、e c .
[0066] is the embedding vector of the target user’s t-th behavior.
[0067] Yes F u The embedding vector of all behaviors of the kth collaborative user in .
[0068] Represents the collection of all feature fields from target items, user profiles, and contextual facets.
[0069] S3. Extract multiple interests of target users:
[0070] Use self-interest as a query to capture the unified interest of each user. The input of the self-attention module consists of query, key, and value, and these three components come from the same place. Multi-head self-attention is a combination of multiple self-attention structures, which can learn relationships in different representation subspaces. Therefore, in this embodiment, multi-head self-attention is used to refine the item representation:
[0071]
[0072] in is the u-th behavior embedding; and is the weight matrix of the u-th head; Q, K, and V are query, key, and value respectively; d u are the dimensions of query, key, and value; in order to capture the similarity in different representation subspaces, the number of heads is set to N, and the multi-head calculation results are connected to form a refined representation of each user behavior sequence:
[0073]
[0074] The correlation between user behavior and other features is measured through the attention mechanism to reflect the user's preference for the target item.
[0075] The user behavior is interacted with all the fields in the attention unit (i.e., all available fields from the target item, user profile, and contextual aspects) to extract the target user interest. However, in real-world industrial systems, the number of fields is large, and including all fields as queries may increase the computational cost of the model. To solve the above problem, a weight is assigned To model the interaction of each field pair between the behavior field and the query field, thereby reducing the noise impact caused by irrelevant fields, the formula is as follows:
[0076]
[0077] in is the u-th user behavior embedding, The target user's interests can be expressed as follows:
[0078]
[0079] S4, Collaborative User Multi-Interest Extraction Module: The multi-head attention layer extracts the relationship between items in the same user to effectively learn and refine the user interest representation:
[0080]
[0081] in is u m User behavior embedding; and is the weight matrix of the u-th head; Q, K, and V are query, key, and value respectively; d u are the dimensions of query, key, and value; in order to capture the similarity in different representation subspaces, the multi-head calculation results are connected and then input into the feedforward network to obtain the interest representation of each collaborative user:
[0082] r n =FFN(concat(head1,head2,…,head N )W O )
[0083] Where N is the number of heads, FN(·) is the feedforward network, and W O is a linear matrix.
[0084] Learning adaptive weights for each similar interest of collaborative users, so as to aggregate these similar interests through the learned weights to obtain the final collaborative user interests; including:
[0085] The attention mechanism is used to calculate the weight of collaborative user interests, and then obtain similar user interests:
[0086]
[0087] in, b a is the learning parameter; a n represents the similarity weight between the target user and the collaborative user; L i is the item behavior length of target item i.
[0088] The interest of collaborative users is obtained by summing the similarity weights between each collaborative user and the target user:
[0089]
[0090] S5. Model training:
[0091] The vector representations of target user interests, collaborative user interests, target items, user profiles, and contexts are concatenated. They are then fed into the MLP to predict the target user's click probability for the target item:
[0092] input = concat(Interest u ,Interest r ,e a ,e i ,e c )
[0093]
[0094] During model training, the following cross entropy loss is minimized:
[0095]
[0096] Where N is the number of training samples, Θ is all trainable parameters, and y i ∈{0,1} represents the label, and λ||Θ||2 represents the L2 regularization term.
[0097] The area under the ROC curve (AUC) and Logloss are widely used as evaluation indicators to evaluate the performance of CTR prediction models. Comparative experimental results of the existing method and the present invention on the Alibaba dataset. The average relative improvement achieved by the present invention in terms of AUC is 1.54%. This further shows that the present invention improves the accuracy of predicting the click rate of the target user for the target item.
[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A click rate prediction method based on user's multiple interests, characterized in that: include: Obtain target user profiles, target user behaviors, target projects, target project contexts, and collaborative user behaviors; Build a click-through rate prediction model based on user's multiple interests; The acquired data is converted into a low-dimensional dense vector through the data embedding layer; The model includes a target user multi-interest extraction module, a collaborative user multi-interest extraction module, and a click rate prediction module; The target user multi-interest extraction module uses the target user behavior vector to obtain a refined target user behavior sequence representation through a multi-head self-attention mechanism; models the interaction of each field pair between the behavior field and the query field to obtain the target user's interests; The collaborative user multi-interest extraction module uses the collaborative user behavior vector to obtain the multi-interest of each collaborative user through a multi-head self-attention machine and a feedforward network; aggregates the multi-interest of each collaborative user to obtain the interest of each collaborative user; The click rate prediction module connects the vector representations of the target user's multiple interests, each collaborative user's interests, the target project, the user profile, and the context; inputs the connected vector into the MLP to predict the target user's click probability on the target project.
2. A click rate prediction method based on user's multiple interests according to claim 1, characterized in that: The data embedding layer is used to convert the acquired data into a low-dimensional dense vector, including: Each feature is encoded into a one-hot vector via a high-dimensional sparse binary encoding and converted to a low-dimensional dense vector: Target user behavior V u , Collaborative User Behavior m , user profile A, target item I and target item context C, the corresponding embedding vectors are X u , X m 、e a 、e i and e c ; is the embedding vector of the target user’s t-th behavior; Yes F u The embedding vector of all behaviors of the kth collaborative user in ; Among them, the collaborative user behavior of user u can be defined as is the user behavior of the t-th collaborative user; Represents the collection of all feature fields from the target project, user profile, and target project context aspects.
3. The method for predicting click rate based on user's multiple interests according to claim 1, characterized in that: The method of using the target user behavior vector to obtain a refined target user behavior sequence representation through a multi-head self-attention mechanism includes: Set the number of heads to N, and connect the multi-head calculation results to form a detailed representation of each user behavior sequence: in, is the u-th behavior embedding; and is the weight matrix of the u-th head; Q, K, and V are query, key, and value respectively; d u It is the dimension of query, key, and value.
4. The method for predicting click rate based on user's multiple interests according to claim 1, characterized in that: The interaction between each field pair between the behavior field and the query field is modeled to obtain the interests of the target user, including: in, is the u-th user behavior embedding; Indicates weight; Interest u Indicates the multiple interests of the target user.
5. The method for predicting click rate based on user's multiple interests according to claim 1, characterized in that: The method of using the collaborative user behavior vector to obtain multiple interests of each collaborative user through a multi-head self-attention machine and a feedforward network includes: The multi-head attention layer extracts the relationship between each collaborative user's items and learns to refine the user interest representation: in, is u m User behavior embedding; and is the weight matrix of the u-th head; Q, K, and V are query, key, and value respectively; d u are the dimensions of query, key, and value; Connect the multi-head computation results and input them into the feed-forward network to obtain the interest representation of each collaborative user: r n =FFN(concat(head1,head2,…,head N )W O ) Where N is the number of heads, FFN(·) is the feedforward network, and W O is a linear matrix.
6. The method for predicting click rate based on user's multiple interests according to claim 1, characterized in that: The aggregating multiple interests of each collaborative user to obtain each collaborative user's interest includes: in, b a is the learning parameter; a n represents the similarity weight between the target user and the collaborative user; L i is the item behavior length of target item i.
7. The method for predicting click rate based on user's multiple interests according to claim 1, characterized in that: The concatenated vector is input into the MLP to predict the click probability of the target user on the target item: input=concat(Interest u ,Interest r ,e a ,e i ,e c ) Among them, e a 、e i and e c are the embedding vectors of user profile A, target item I, and target item context C.
8. The method for predicting click rate based on user's multiple interests according to claim 1, characterized in that: The click rate prediction loss function based on user's multiple interests is: Where N is the number of training samples; Θ is all trainable parameters; y i ∈{0,1} represents the label; λ||Θ||2 represents the L2 regularization term.