Click-through rate prediction method and system based on knowledge enhancement and contrastive learning

Abstract

The present invention relates to a click rate prediction method and system based on knowledge enhancement and contrastive learning, the method comprising the following steps: step A: collecting user behavior data, including user ID, product ID, product type and interaction time, and constructing a click rate prediction training set; step B: using the training set to train a deep learning network model based on knowledge enhancement and contrastive learning; step C: inputting user and product data into the trained deep learning network model, and outputting the current user's click probability for the candidate product. The method and system are conducive to improving the accuracy of click rate prediction.

Description

Technical Field

[0001] The present invention belongs to the field of recommendation systems, and in particular relates to a click rate prediction method and system based on knowledge enhancement and contrastive learning. Background Art

[0002] Click-through rate prediction is the core part of the recommendation system. This part can predict the possible click probability of any user on a certain item, and further sort the click probability from high to low, so as to generate a list of candidate items that the user may be interested in, cater to the user's preferences, avoid the user's direct contact with massive information, and help the user find valuable key content. In click-through rate prediction, user portraits and a large number of user behavior logs, including browsing, adding to shopping carts, purchasing, and evaluating behaviors, imply the user's interests. Therefore, behavior modeling is an important topic in click-through rate prediction. In recent years, with the development of deep learning, the rapid growth of user historical behavior data has given birth to a number of excellent click-through rate prediction models. A large amount of user behavior information brings information gain, but how to effectively deal with the long-term historical behavior sequence of users with a lot of noise is also a problem that needs to be solved. The rich behavior data also brings many challenges to the modeling of click-through rate prediction tasks. How to use these behavior data to improve the performance of click-through rate prediction models has become a hot research direction.

[0003] However, although existing models have achieved performance improvements to some extent, they still face some major weaknesses. Most importantly, they lack concrete semantics and ignore the root causes that drive users to click on target items. Therefore, they cannot accurately understand users' interests. Knowledge graphs can flexibly model comprehensive auxiliary data and store external heterogeneous knowledge in the form of head entities, relations, and tail entities. Due to its rich information, knowledge graphs can be combined to enrich the representation of users and items and improve the interpretability of recommendations. Despite great improvements, it is still challenging to effectively integrate such heterogeneous information for recommendation. Existing knowledge graph-based recommendation methods are mainly divided into two categories, path-based methods and graph neural network-based methods. Path-based methods infer user preferences by exploring multiple meta-paths of target user-item pairs on knowledge graphs, which usually requires domain knowledge. More importantly, this type of method ignores the rich structural information implicit in the knowledge graph and therefore cannot fully characterize the potential relationship between a given target user and item. Graph neural network-based methods explicitly model high-order connectivity in knowledge graphs to provide strong performance. However, these methods mainly enrich the representations of target users and items respectively by aggregating their original neighbors in the KG, without considering the historical behavior of the target user. Summary of the invention

[0004] The purpose of the present invention is to provide a click-through rate prediction method and system based on knowledge enhancement and contrastive learning, which method and system are conducive to improving the accuracy of click-through rate prediction.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: a click rate prediction method based on knowledge enhancement and contrastive learning, comprising the following steps:

[0006] Step A: Collect user behavior data, including user ID, product ID, product type, and interaction time, and build a click-through rate prediction training set;

[0007] Step B: Use the training set to train the deep learning network model G based on knowledge enhancement and contrastive learning;

[0008] Step C: Input the user and product data into the trained deep learning network model G, and output the click probability of the current user for the candidate product.

[0009] Furthermore, the step B specifically includes the following steps:

[0010] Step B1: Construct the knowledge graph G kg And product co-occurrence graph G cf ; Each sample in the training set contains the user ID, the candidate product ID and the user's behavior sequence; a sample in the training set is mapped according to the knowledge graph G kg Segment and obtain N candidate segments p candi ={p1,p2,...,p N}, and then through the embedding layer, the vector representation e of the candidate segment is obtained candi ={e1,e2,...,e i};

[0011] Step B2: The vector representation e of the candidate segment obtained in step B1 candi Perform a double-layer attention mechanism to obtain the knowledge-enhanced user interest vector representation f u ;

[0012] Step B3: The candidate segment p obtained in step B1 candi Divided into strong interest segments strong and weak interest segment p weak , and then through the embedding layer, we get the vector representation e of the strong interest segment strong ={e 1,strong ,e 2,strong ,...,e i,strong} and the vector representation of the weak interest segment e weak ={e 1,weak ,e 2,weak ,...,e i,weak}; The target product vector is represented by et As a query, the vector representation e of the strong interest segment strong Perform a two-layer attention mechanism to obtain a strong interest vector representation f strong ;

[0013] Step B4: In the knowledge graph G kg And product co-occurrence graph G cf In the process, based on each product ID and candidate product ID in the sample, a knowledge graph potential interest segment is constructed. Potential interest segments that co-occur with products Then, after the embedding layer, we get the vector representation of the potential interest segment of the knowledge graph. Vector representation of potential interest segments that co-occur with products

[0014] Step B5: Represent the strong interest vector obtained in step B3 as f strong and the target product vector representation e t Perform concatenation, as a query, on the vector representation of the weak interest segment e weak Perform a double-layer attention mechanism to obtain a weak interest vector representation f weak ;

[0015] Step B6: Represent the target product vector e t As a query, a vector representation of the potential segment of interest and Perform a two-layer attention mechanism to obtain a vector representation of the potential interest vector and right and A contrast mechanism is implemented to capture the complementary and distinguishing information between two potential interest segments; the interests of the same user from two segments are regarded as a pair of positive samples, and the interests of different users are regarded as paired negative samples, and the contrast loss L is obtained. cl ;

[0016] Step B7: The knowledge-enhanced user interest vector obtained in steps B2, B3, and B6 is represented as f u , strong interest vector representation f strong , weak interest vector representation f weak , potential interest vector representation and Splice them together and input them into a multi-layer perceptron for click-through rate prediction. The predicted click-through rate is obtained and the prediction loss is calculated using the cross entropy function. Then, according to the target loss function, the gradient of each parameter in the deep learning network model is calculated by the back propagation method, and the parameters are updated using the stochastic gradient descent method.

[0017] Step B8: When the loss value generated by the deep learning network model is less than the set threshold or reaches the maximum number of iterations, the training of the deep learning network model G is terminated.

[0018] Furthermore, the step B1 specifically includes the following steps:

[0019] Step B11: Use Freebase to build a knowledge graph G for all products kg , if a mapping is available, the item is mapped to the Freebase entity by title matching, and there is an edge between the item and the entity if there is some relationship between them;

[0020] Step B12: Construct a product co-occurrence graph G based on the interaction between users and products cf ,If two products appear at the same time in the user’s behavior sequence, there is an edge between the two products;

[0021] Step B13: Based on the knowledge graph G obtained in step B11 kg , store the products in the sample that have edges with the same entity in one segment, traverse the single user sequence, and obtain N candidate segments p candi ={p1,p2,...,p N}; Then, after passing through the embedding layer, the vector representation of the candidate segment is obtained candi ={e1,e2,...,e i}.

[0022] Furthermore, the step B2 specifically includes the following steps:

[0023] Step B21: Use the target product vector to represent e t As a query, for each product vector in a segment, we represent e i Do the attention calculation and get a segment vector representation f seg :

[0024] α i =σ(W1(W2([e i ,e t ])))

[0025]

[0026] Among them, α i represents the attention coefficient, σ() represents the sigmoid function, W1 and W2 represent trainable parameters, and n p Indicates the number of products in the segment;

[0027] Step B22: Use the target product vector to represent e tAs a query, the vector representation of each segment Perform attention calculation to obtain the knowledge-enhanced user interest vector representation f u :

[0028]

[0029] Among them, β j represents the attention coefficient, W3 and W4 represent trainable parameters, n seg Indicates the number of segments.

[0030] Furthermore, the step B3 specifically includes the following steps:

[0031] Step B31: The candidate segment p obtained in step B1 is candi Segmentation is performed based on the number of products in the segment, where the number of products greater than 1 is considered a strong interest segment. strong , the rest are regarded as weak interest segments p weak ;

[0032] Step B32: Use the target product vector to represent e t As a query, for each product vector in a strong interest segment, e i,strong Perform attention calculation to obtain a vector representation f of the segment of strong interest seg,strong :

[0033] α i,strong =σ(W 1,strong (W 2,strong ([e i,strong ,e t ])))

[0034]

[0035] Among them, α i,strong represents the attention coefficient, W 1,strong and W 2,strong represents the trainable parameters, n p,strong Indicates the number of products in the strong interest segment;

[0036] Step B33: Use the target product vector to represent e t As a query, the vector representation of each strong interest segment Perform attention calculation to obtain a strong interest vector representation f strong :

[0037]

[0038] Among them, β j,strong represents the attention coefficient, W 3,strong and W 4,strong represents the trainable parameters, nseg,strong Indicates the number of strong interest segments.

[0039] Furthermore, the step B4 specifically includes the following steps:

[0040] Step B41: In the knowledge graph G kg In the example, find the shortest path between each product ID in the sample and the candidate product ID, remove the entity ID that appears in the path, and add the nodes that appear in the path to the potential interest segment. middle;

[0041] Step B42: In the product co-occurrence graph G cf In the example, find the shortest path between each product ID in the sample and the candidate product ID, and add the nodes that appear in the path to the potential interest segment middle.

[0042] Furthermore, the step B5 specifically includes the following steps:

[0043] Step B51: Represent the strong interest vector obtained in step B3 as f strong and the target product vector representation e t Splice and use it as a query to represent each product vector in a weak interest segment. i,weak Perform attention calculation to obtain a vector representation f of the weak interest segment seg,weak :

[0044] α i,weak =σ(W 1,weak (W 2,weak ([e i,weak ,f strong ,e t ])))

[0045]

[0046] Among them, α i,weak represents the attention coefficient, W 1,weak and W 2,weak represents the trainable parameters, n p,weak Indicates the number of products in the weak interest segment;

[0047] Step B52: Use the target product vector to represent e t As a query, the vector representation of each weak interest segment Perform attention calculation to obtain the representation of the weak interest vector f weak :

[0048]

[0049] Among them, β j,weakrepresents the attention coefficient, W 3,weak and W 4,weak represents the trainable parameters, n seg,weak Indicates the number of weak interest segments.

[0050] Furthermore, the step B6 specifically includes the following steps:

[0051] Step B61: Use the target product vector to represent e t As a query, each product vector in a potential interest segment of a knowledge graph represents Perform attention calculation to obtain the vector representation of the potential interest segment of the knowledge graph

[0052]

[0053] in, represents the attention coefficient, and represents the trainable parameters, Indicates the number of products in the potential interest segment of the knowledge graph;

[0054] Step B62: Use the target product vector to represent e t As a query, the vector representation of each knowledge graph potential interest segment Perform attention calculation to obtain the representation of the potential interest vector of the knowledge graph

[0055]

[0056] in, represents the attention coefficient, and represents the trainable parameters, Indicates the number of potential interest segments in the knowledge graph;

[0057] Step B63: Use the target product vector to represent e t As a query, each product vector in a product co-occurrence potential interest segment represents Perform attention calculation to obtain the vector representation of the potential interest segment of the co-occurrence of products

[0058]

[0059] in, represents the attention coefficient, and represents the trainable parameters, Indicates the number of products that co-occur in the potential interest segment;

[0060] Step B64: Use the target product vector to represent e t As a query, the vector representation of the potential interest segment for each product co-occurrence Perform attention calculation to obtain the representation of weak interest vector

[0061]

[0062] in, represents the attention coefficient, and represents the trainable parameters, Indicates the number of potential interest segments where products co-occur;

[0063] Step B65: Take the interests of the same user from two segments as a pair of positive samples, and the interests of different users as a pair of negative samples, and calculate the contrast loss L cl :

[0064]

[0065] Where sim(·) represents the cosine similarity function, τ is the temperature coefficient, and n user Indicates the number of users.

[0066] Furthermore, the step B7 specifically includes the following steps:

[0067] Step B71: Represent the user's vector e u , vector representation of candidate products e t And the knowledge-enhanced user interest vector representation f obtained in steps B2, B3, and B6 u , strong interest vector representation f strong , the representation of weak interest vector f weak , vector representation of potential interest vector and Splice them together and input them into the multi-layer perceptron MLP for click-through rate prediction to get the predicted click-through rate:

[0068]

[0069] Step B72: Use the cross entropy function L target Calculate the prediction loss, update the learning rate through the gradient optimization algorithm Adam, and use back propagation to iteratively update the model parameters to minimize the loss function to train the model; the total model loss is the weighted addition of the above losses:

[0070]

[0071] L all =L tar get +μ1Lcl

[0072] Among them, y i is the corresponding label, μ1 is the hyperparameter, L tar get is the loss function for click rate prediction, L cl is the contrastive loss function of the latent interest vector.

[0073] The present invention also provides a click rate prediction system using the above method, comprising:

[0074] The training set construction module is used to collect user behavior data and construct a training set;

[0075] A model training module, used for training a deep learning network model G based on knowledge enhancement and contrastive learning; and

[0076] The click rate prediction module is used to receive user and product data and input them into the trained deep learning network model G, and output the click probability of the current user for the candidate product.

[0077] Compared with the prior art, the present invention has the following beneficial effects: first, based on the knowledge graph, the present invention divides the user's behavior sequence into strong interest segments, weak interest segments and potential interest segments, introduces semantic information, and extracts the user's interests from multiple aspects. At the same time, it introduces a strong-to-weak attention mechanism, uses strong interests to guide the extraction of weak interests, and can more accurately extract the user's interests. In addition, by contrasting learning tasks to capture the complementary relationship between the two potential interest vectors obtained from the knowledge graph and the product co-occurrence graph, the user's interests can be effectively expanded, and products that do not interact but meet the user's interests can be introduced to the user, so that the model can learn more interactive information. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 is a flow chart of a method implementation of an embodiment of the present invention;

[0079] Figure 2 is an architecture diagram of a deep learning network model in an embodiment of the present invention;

[0080] Figure 3 Schematic diagram of the system structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0081] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0082] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0083] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0084] like Figure 1 As shown, this embodiment provides a click rate prediction method based on knowledge enhancement and contrastive learning, comprising the following steps:

[0085] Step A: Collect user behavior data, including user ID, product ID, product type, and interaction time, and build a click-through rate prediction training set.

[0086] Step B: Use the training set to train the deep learning network model G based on knowledge enhancement and contrastive learning.

[0087] Step C: Input the user and product data into the trained deep learning network model G, and output the click probability of the current user for the candidate product.

[0088] In this embodiment, the architecture of the deep learning network model G based on knowledge enhancement and contrastive learning is as follows: Figure 2 shown.

[0089] In this embodiment, step B specifically includes the following steps:

[0090] Step B1: Construct the knowledge graph G kg And product co-occurrence graph G cf ; Each sample in the training set contains the user ID, the candidate product ID and the user's behavior sequence; a sample in the training set is mapped according to the knowledge graph G kg Segment and obtain N candidate segments p candi ={p1,p2,...,p N}, and then through the embedding layer, the vector representation e of the candidate segment is obtained candi ={e1,e2,...,e i}.

[0091] In this embodiment, step B1 specifically includes the following steps:

[0092] Step B11: Use Freebase to build a knowledge graph G for all products kg , if a mapping is available, the item is mapped to the Freebase entity by title matching, and there is an edge between the item and the entity if there is some relationship between them;

[0093] Step B12: Construct a product co-occurrence graph G based on the interaction between users and products cf ,If two items appear at the same time in the user’s behavior sequence, there is an edge between the two items.

[0094] Step B13: Based on the knowledge graph G obtained in step B11 kg , store the products in the sample that have edges with the same entity in one segment, traverse the single user sequence, and obtain N candidate segments p candi ={p1,p2,...,p N}; Then, after passing through the embedding layer, the vector representation of the candidate segment is obtained candi ={e1,e2,...,e i}.

[0095] Step B2: The vector representation e of the candidate segment obtained in step B1 candi Perform a double-layer attention mechanism to obtain the knowledge-enhanced user interest vector representation f u .

[0096] In this embodiment, step B2 specifically includes the following steps:

[0097] Step B21: Use the target product vector to represent e t As a query, for each product vector in a segment, we represent e i Do the attention calculation and get a segment vector representation f seg :

[0098] α i =σ(W1(W2([e i ,e t ])))

[0099]

[0100] Among them, α i represents the attention coefficient, σ() represents the sigmoid function, W1 and W2 represent trainable parameters, and n p Indicates the number of products in the segment.

[0101] Step B22: Use the target product vector to represent e t As a query, the vector representation of each segment Perform attention calculation to obtain the knowledge-enhanced user interest vector representation f u :

[0102]

[0103] Among them, β j represents the attention coefficient, W3 and W4 represent trainable parameters, nseg Indicates the number of segments.

[0104] Step B3: The candidate segment p obtained in step B1 candi Divided into strong interest segments strong and weak interest segment p weak , and then through the embedding layer, we get the vector representation e of the strong interest segment strong ={e 1,strong ,e 2,strong ,...,e i,strong} and the vector representation of the weak interest segment e weak ={e 1,weak ,e 2,weak ,...,e i,weak}; The target product vector is represented by e t As a query, the vector representation e of the strong interest segment strong Perform a two-layer attention mechanism to obtain a strong interest vector representation f strong .

[0105] In this embodiment, step B3 specifically includes the following steps:

[0106] Step B31: The candidate segment p obtained in step B1 is candi Segmentation is performed based on the number of products in the segment, where the number of products greater than 1 is considered a strong interest segment. strong , the rest are regarded as weak interest segments p weak .

[0107] Step B32: Use the target product vector to represent e t As a query, for each product vector in a strong interest segment, e i,strong Perform attention calculation to obtain a vector representation f of the segment of strong interest seg,strong :

[0108] α i,strong =σ(W 1,strong (W 2,strong ([e i,strong ,e t ])))

[0109]

[0110] Among them, α i,strong represents the attention coefficient, W 1,strong and W 2,strong represents the trainable parameters, n p,strong Indicates the number of products in the strong interest segment.

[0111] Step B33: Use the target product vector to represent e t As a query, the vector representation of each strong interest segment Perform attention calculation to obtain a strong interest vector representation f strong :

[0112]

[0113] Among them, β j,strong represents the attention coefficient, W 3,strong and W 4,strong represents the trainable parameters, n seg,strong Indicates the number of strong interest segments.

[0114] Step B4: In the knowledge graph G kg And product co-occurrence graph G cf In the process, based on each product ID and candidate product ID in the sample, a knowledge graph potential interest segment is constructed. Potential interest segments that co-occur with products Then, after the embedding layer, we get the vector representation of the potential interest segment of the knowledge graph. Vector representation of potential interest segments that co-occur with products

[0115] In this embodiment, step B4 specifically includes the following steps:

[0116] Step B41: In the knowledge graph G kg In the example, find the shortest path between each product ID in the sample and the candidate product ID, remove the entity ID that appears in the path, and add the nodes that appear in the path to the potential interest segment. middle.

[0117] Step B42: In the product co-occurrence graph G cf In the example, find the shortest path between each product ID in the sample and the candidate product ID, and add the nodes that appear in the path to the potential interest segment middle.

[0118] Step B5: Represent the strong interest vector obtained in step B3 as f strong and the target product vector representation e t Perform concatenation, as a query, on the vector representation of the weak interest segment e weak Perform a double-layer attention mechanism to obtain a weak interest vector representation f weak ;

[0119] In this embodiment, step B5 specifically includes the following steps:

[0120] Step B51: Represent the strong interest vector obtained in step B3 as f strong and the target product vector representation e tSplice and use it as a query to represent each product vector in a weak interest segment. i,weak Perform attention calculation to obtain a vector representation f of the weak interest segment seg,weak :

[0121] α i,weak =σ(W 1,weak (W 2,weak ([e i,weak ,f strong ,e t ])))

[0122]

[0123] Among them, α i,weak represents the attention coefficient, W 1,weak and W 2,weak represents the trainable parameters, n p,weak Indicates the number of products in the weak interest segment.

[0124] Step B52: Use the target product vector to represent e t As a query, the vector representation of each weak interest segment Perform attention calculation to obtain the representation of the weak interest vector f weak :

[0125]

[0126] Among them, β j,weak represents the attention coefficient, W 3,weak and W 4,weak represents the trainable parameters, n seg,weak Indicates the number of weak interest segments.

[0127] Step B6: Represent the target product vector e t As a query, a vector representation of the potential segment of interest and Perform a two-layer attention mechanism to obtain a vector representation of the potential interest vector and right and A contrast mechanism is implemented to capture the complementary and distinguishing information between two potential interest segments; the interests of the same user from two segments are regarded as a pair of positive samples, and the interests of different users are regarded as paired negative samples, and the contrast loss L is obtained. cl .

[0128] In this embodiment, step B6 specifically includes the following steps:

[0129] Step B61: Use the target product vector to represent e tAs a query, each product vector in a potential interest segment of a knowledge graph represents Perform attention calculation to obtain the vector representation of the potential interest segment of the knowledge graph

[0130]

[0131] in, represents the attention coefficient, and represents the trainable parameters, Indicates the number of products in the potential interest segment of the knowledge graph.

[0132] Step B62: Use the target product vector to represent e t As a query, the vector representation of each knowledge graph potential interest segment Perform attention calculation to obtain the representation of the potential interest vector of the knowledge graph

[0133]

[0134] in, represents the attention coefficient, and represents the trainable parameters, Represents the number of potential interest segments in the knowledge graph.

[0135] Step B63: Use the target product vector to represent e t As a query, each product vector in a product co-occurrence potential interest segment represents Perform attention calculation to obtain the vector representation of the potential interest segment of the co-occurrence of products

[0136]

[0137] in, represents the attention coefficient, and represents the trainable parameters, Indicates the number of products that co-occur in the potential interest segment.

[0138] Step B64: Use the target product vector to represent e t As a query, the vector representation of the potential interest segment for each product co-occurrence Perform attention calculation to obtain the representation of weak interest vector

[0139]

[0140] in, represents the attention coefficient, and represents the trainable parameters, Indicates the number of potential interest segments where products co-occur.

[0141] Step B65: Take the interests of the same user from two segments as a pair of positive samples, and the interests of different users as a pair of negative samples, and calculate the contrast loss L cl :

[0142]

[0143] Where sim(·) represents the cosine similarity function, τ is the temperature coefficient, and n user Indicates the number of users.

[0144] Step B7: The knowledge-enhanced user interest vector obtained in steps B2, B3, and B6 is represented as f u , strong interest vector representation f strong , weak interest vector representation f weak , potential interest vector representation and The data are spliced ​​together and input into a multi-layer perceptron for click-through rate prediction to obtain the predicted click-through rate. The prediction loss is calculated using the cross entropy function. Then, based on the target loss function, the gradients of the parameters in the deep learning network model are calculated using the back propagation method, and the parameters are updated using the stochastic gradient descent method.

[0145] In this embodiment, step B7 specifically includes the following steps:

[0146] Step B71: Represent the user's vector e u , vector representation of candidate products e t And the knowledge-enhanced user interest vector representation f obtained in steps B2, B3, and B6 u , strong interest vector representation f strong , the representation of weak interest vector f weak , vector representation of potential interest vector and Splice them together and input them into the multi-layer perceptron MLP for click-through rate prediction to get the predicted click-through rate:

[0147]

[0148] Step B72: Use the cross entropy function L target Calculate the prediction loss, update the learning rate through the gradient optimization algorithm Adam, and use back propagation to iteratively update the model parameters to minimize the loss function to train the model; the total model loss is the weighted addition of the above losses:

[0149]

[0150] L all =L target +μ1L cl

[0151] Among them, y i is the corresponding label, μ1 is the hyperparameter, L target is the loss function for click rate prediction, L cl is the contrastive loss function of the latent interest vector.

[0152] Step B8: When the loss value generated by the deep learning network model is less than the set threshold or reaches the maximum number of iterations, the training of the deep learning network model G is terminated.

[0153] like Figure 3 As shown, this embodiment also provides a click rate prediction system using the above method, including: a training set construction module, a model training module and a click rate prediction module.

[0154] The training set construction module is used to collect user behavior data and construct a training set.

[0155] The model training module is used to train a deep learning network model G based on knowledge enhancement and contrastive learning.

[0156] The click rate prediction module is used to receive user and product data and input them into the trained deep learning network model G, and output the click probability of the current user on the candidate product.

[0157] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0158] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0159] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0161] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A click rate prediction method based on knowledge enhancement and contrastive learning, characterized in that: The following steps are involved: Step A: Collect user behavior data, including user ID, product ID, product type, and interaction time, and build a click-through rate prediction training set; Step B: Use the training set to train the deep learning network model G based on knowledge enhancement and contrastive learning; Step C: Input the user and product data into the trained deep learning network model G, and output the click probability of the current user for the candidate product; The step B specifically comprises the following steps: Step B1: Construct the knowledge graph G kg And product co-occurrence graph G cf ; Each sample in the training set contains the user ID, the candidate product ID and the user's behavior sequence; a sample in the training set is mapped according to the knowledge graph G kg Segment and obtain N candidate segments p candi ={p1,p2,...,p N }, and then through the embedding layer, the vector representation e of the candidate segment is obtained candi ={e1,e2,...,e i }; Step B2: The vector representation e of the candidate segment obtained in step B1 candi Perform a double-layer attention mechanism to obtain the knowledge-enhanced user interest vector representation f u ; Step B3: The candidate segment p obtained in step B1 candi Divided into strong interest segments strong and weak interest segment p weak , and then through the embedding layer, we get the vector representation e of the strong interest segment strong ={e 1,strong ,e 2,strong ,...,e i,strong } and the vector representation of the weak interest segment e weak ={e 1,weak ,e 2,weak ,...,e i,weak }; The target product vector is represented by e t As a query, the vector representation e of the strong interest segment strong Perform a two-layer attention mechanism to obtain a strong interest vector representation f strong ; Step B4: In the knowledge graph G kg And product co-occurrence graph G cf In the process, based on each product ID and candidate product ID in the sample, a knowledge graph potential interest segment is constructed. Potential interest segments that co-occur with products Then, after the embedding layer, we get the vector representation of the potential interest segment of the knowledge graph. Vector representation of potential interest segments that co-occur with products Step B5: Represent the strong interest vector obtained in step B3 as f strong and the target product vector representation e t Perform concatenation, as a query, on the vector representation of the weak interest segment e weak Perform a double-layer attention mechanism to obtain a weak interest vector representation f weak ; Step B6: Represent the target product vector e t As a query, a vector representation of the potential segment of interest and Perform a two-layer attention mechanism to obtain a vector representation of the potential interest vector and right and A contrast mechanism is implemented to capture the complementary and distinguishing information between two potential interest segments; the interests of the same user from two segments are regarded as a pair of positive samples, and the interests of different users are regarded as paired negative samples, and the contrast loss L is obtained. cl ; Step B7: The knowledge-enhanced user interest vector obtained in steps B2, B3, and B6 is represented as f u , strong interest vector representation f strong , weak interest vector representation f weak , potential interest vector representation and Splice them together and input them into a multi-layer perceptron for click-through rate prediction. The predicted click-through rate is obtained and the prediction loss is calculated using the cross entropy function. Then, according to the target loss function, the gradient of each parameter in the deep learning network model is calculated by the back propagation method, and the parameters are updated using the stochastic gradient descent method. Step B8: When the loss value generated by the deep learning network model is less than the set threshold or reaches the maximum number of iterations, the training of the deep learning network model G is terminated.

2. The click rate prediction method based on knowledge enhancement and contrastive learning according to claim 1 is characterized in that: The step B1 specifically comprises the following steps: Step B11: Use Freebase to build a knowledge graph G for all products kg , if a mapping is available, the item is mapped to the Freebase entity by title matching, and there is an edge between the item and the entity if there is some relationship between them; Step B12: Construct a product co-occurrence graph G based on the interaction between users and products cf ,If two products appear at the same time in the user’s behavior sequence, there is an edge between the two products; Step B13: Based on the knowledge graph G obtained in step B11 kg , store the products in the sample that have edges with the same entity in one segment, traverse the single user sequence, and obtain N candidate segments p candi ={p1,p2,...,p N }; Then, after passing through the embedding layer, the vector representation of the candidate segment is obtained candi ={e1,e2,...,e i }.

3. The click rate prediction method based on knowledge enhancement and contrastive learning according to claim 2 is characterized in that: The step B2 specifically comprises the following steps: Step B21: Use the target product vector to represent e t As a query, for each product vector in a segment, we represent e i Do the attention calculation and get a segment vector representation f seg : a i =σ(W1(W2([e i ,and t ]))) Among them, α i represents the attention coefficient, σ() represents the sigmoid function, W1 and W2 represent trainable parameters, and n p Indicates the number of products in the segment; Step B22: Use the target product vector to represent e t As a query, the vector representation of each segment Perform attention calculation to obtain the knowledge-enhanced user interest vector representation f u : Among them, β j represents the attention coefficient, W3 and W4 represent trainable parameters, n seg Indicates the number of segments.

4. The click rate prediction method based on knowledge enhancement and contrastive learning according to claim 3 is characterized in that: The step B3 specifically comprises the following steps: Step B31: The candidate segment p obtained in step B1 is candi Segmentation is performed based on the number of products in the segment, where the number of products greater than 1 is considered a strong interest segment. strong , the rest are considered as weak interest segments p weak ; Step B32: Use the target product vector to represent e t As a query, for each product vector in a strong interest segment, e i,strong Perform attention calculation to obtain a vector representation f of the segment of strong interest seg,strong : a i,strong =σ(t 1,strong (W 2,strong ([e i,strong ,e t ]))) Among them, α i,strong represents the attention coefficient, W 1,strong and W 2,strong represents the trainable parameters, n p,strong Indicates the number of products in the strong interest segment; Step B33: Use the target product vector to represent e t As a query, the vector representation of each strong interest segment Perform attention calculation to obtain a strong interest vector representation f strong : Among them, β j,strong represents the attention coefficient, W 3,strong and W 4,strong represents the trainable parameters, n seg,strong Indicates the number of strong interest segments.

5. The click rate prediction method based on knowledge enhancement and contrastive learning according to claim 4 is characterized in that: The step B4 specifically comprises the following steps: Step B41: In the knowledge graph G kg In the example, find the shortest path between each product ID in the sample and the candidate product ID, remove the entity ID that appears in the path, and add the nodes that appear in the path to the potential interest segment. middle; Step B42: In the product co-occurrence graph G cf In the example, find the shortest path between each product ID in the sample and the candidate product ID, and add the nodes that appear in the path to the potential interest segment middle.

6. The click rate prediction method based on knowledge enhancement and contrastive learning according to claim 5 is characterized in that: The step B5 specifically comprises the following steps: Step B51: Represent the strong interest vector obtained in step B3 as f strong and the target product vector representation e i Splice and use it as a query to represent each product vector in a weak interest segment. i,weak Perform attention calculation to obtain a vector representation f of the weak interest segment seg,weak : a i,weak =σ(W 1,weak (W 2,weak ([e i,weak ,f strong ,e t ]))) Among them, α i,weak represents the attention coefficient, W 1,weak and W 2,weak represents the trainable parameters, n p,weak Indicates the number of products in the weak interest segment; Step B52: Use the target product vector to represent e t As a query, the vector representation of each weak interest segment Perform attention calculation to obtain the representation of the weak interest vector f weak : Among them, β j,weak represents the attention coefficient, W 3,weak and W 4,weak represents the trainable parameters, n seg,weak Indicates the number of weak interest segments.

7. The click rate prediction method based on knowledge enhancement and contrastive learning according to claim 6 is characterized in that: The step B6 specifically comprises the following steps: Step B61: Use the target product vector to represent e t As a query, each product vector in a potential interest segment of a knowledge graph represents Perform attention calculation to obtain the vector representation of the potential interest segment of the knowledge graph in, represents the attention coefficient, and represents the trainable parameters, Indicates the number of products in the potential interest segment of the knowledge graph; Step B62: Use the target product vector to represent e t As a query, the vector representation of each knowledge graph potential interest segment Perform attention calculation to obtain the representation of the potential interest vector of the knowledge graph in, represents the attention coefficient, and represents the trainable parameters, Indicates the number of potential interest segments in the knowledge graph; Step B63: Use the target product vector to represent e t As a query, each product vector in a product co-occurrence potential interest segment represents Perform attention calculation to obtain the vector representation of the potential interest segment of the co-occurrence of products in, represents the attention coefficient, and represents the trainable parameters, Indicates the number of products that co-occur in the potential interest segment; Step B64: Use the target product vector to represent e t As a query, the vector representation of the potential interest segment for each product co-occurrence Perform attention calculation to obtain the representation of weak interest vector in, represents the attention coefficient, and represents the trainable parameters, Indicates the number of potential interest segments where products co-occur; Step B65: Take the interests of the same user from two segments as a pair of positive samples, and the interests of different users as a pair of negative samples, and calculate the contrast loss L cl : Where sim(·) represents the cosine similarity function, τ is the temperature coefficient, and n user Indicates the number of users.

8. The click rate prediction method based on knowledge enhancement and contrastive learning according to claim 7 is characterized in that: The step B7 specifically includes the following steps: Step B71: Represent the user's vector e u , vector representation of candidate products e t And the knowledge-enhanced user interest vector representation f obtained in steps B2, B3, and B6 u , strong interest vector representation f strong , the representation of weak interest vector f weak , vector representation of potential interest vector and Splice them together and input them into the multi-layer perceptron MLP for click-through rate prediction to get the predicted click-through rate: Step B72: Use the cross entropy function L target Calculate the prediction loss, update the learning rate through the gradient optimization algorithm Adam, and use back propagation to iteratively update the model parameters to minimize the loss function to train the model; the total model loss is the weighted addition of the above losses: L all =L target +μ1L cl Among them, y i is the corresponding label, μ1 is the hyperparameter, L target is the loss function for click rate prediction, L cl is the contrastive loss function of the latent interest vector.

9. A click rate prediction system using the method according to any one of claims 1 to 8, characterized in that: include: The training set construction module is used to collect user behavior data and construct a training set; Model training module, used to train the deep learning network model G based on knowledge enhancement and contrastive learning; as well as The click rate prediction module is used to receive user and product data and input them into the trained deep learning network model G, and output the click probability of the current user for the candidate product.

Images