User preference prediction method based on data depolarization

By using embedded layer, feature filtering layer and factor decomposition machine in user preference prediction, the time deviation problem in video data is solved, the accuracy of user preference prediction is improved, and more accurate users are achieved Preference prediction and recommendation effects.

CN118916761AActive Publication Date: 2024-11-08CHONGQING UNIV OF POSTS & TELECOMM
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411031310.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-08
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

The prior art is difficult to effectively handle the duration deviation in the video dataset in user preference prediction, resulting in the model being unable to capture the user's real interests, and the existing debiasing methods cannot effectively remove the influence of unobserved confusing factors.

Method used

A user preference prediction method based on data debias is proposed. By obtaining video data and user data, using technologies such as embedding layer, feature filtering layer and factor decomposition machine to perform embedding processing, debias processing and feature important weight calculation to generate users Interest represents vector, and finally adjusts the model parameters through cross entropy loss to train an efficient user preference prediction model.

Benefits of technology

It improves the accuracy of user preference prediction, avoids the prediction inaccuracy problem caused by the recommendation model relying too much on initial data, and achieves more accurate user preference prediction through feature cross-combination, which enhances the effect of the recommendation system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916761A_ABST
    Figure CN118916761A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of internet user preference prediction, and particularly relates to a user preference prediction method based on data depolarization. The method comprises the following steps: acquiring video data and user data; processing the video data and the user data by adopting an embedding layer to obtain user embedding and video embedding; performing depolarization processing on the user embedding and the video embedding to obtain depolarized user embedding and depolarized video embedding; processing user embedding, video embedding, debiased user embedding and debiased video embedding by adopting a feature screening layer to obtain a user specific feature important weight and a video specific feature important weight; processing the user specific feature important weight and the video specific feature important weight by adopting a factorization machine to obtain a user interest representation vector; inputting the user interest representation vector into an output layer for processing to obtain a user preference prediction result; according to the invention, user preference prediction can be accurately and effectively carried out, and the accuracy of user preference prediction is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet user preference prediction, and in particular relates to a user preference prediction method based on data debiasing. Background Art

[0002] A recommendation system is a decision support system that can predict a user's interest in a specific item and help the user select the content of interest through information such as historical behavior and preferences. Among them, the purpose of click-through rate prediction is to predict the probability of a user clicking on an advertisement or item, which is crucial for many online applications such as online advertising and recommendation systems. However, existing datasets are usually observed and analyze user interest preferences based on historical interactions, but there are often various biases in such datasets, such as user selection bias for items, system exposure bias for items, and item content bias. Models trained with biased data cannot capture the user's true interests, so mitigating the impact of biased data is the key to improving cache hit rates.

[0003] For user preference prediction, how to capture the user's true interest preferences from a biased data set is an important issue. This issue has been widely studied, but there is still a lack of a method to deal with the duration deviation in the video data set. Among them, drawing a complete causal graph and then using the backdoor adjustment principle to remove the influence of confounding factors has been proven to be an effective debiasing method. However, this method still has its limitations. First of all, the confounding factors drawn by the causal graph are more about removing observable confounding factors, and there is no better way to remove unobservable confounding factors. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes a user preference prediction method based on data debiasing, the method comprising: acquiring video data and user data; inputting the video data and user data into a trained user preference prediction model to obtain a user preference prediction result;

[0005] The training process of the user preference prediction model includes:

[0006] S1: Obtain video data and user data for training; use an embedding layer to process the video data and user data to obtain user embedding and video embedding;

[0007] S2: Debias the user embedding and the video embedding to obtain debiased user embedding and debiased video embedding;

[0008] S3: Process the user embedding, video embedding, debiased user embedding and debiased video embedding using a feature filtering layer to obtain user-specific feature importance weights and video-specific feature importance weights;

[0009] S4: Use a factorization machine to process the user-specific feature importance weights and the video-specific feature importance weights to obtain a user interest representation vector;

[0010] S5: Input the user interest representation vector into the output layer for processing to obtain the user preference prediction result;

[0011] S6: Calculate the model cross entropy loss and adjust the model parameters according to the model cross entropy loss to obtain a trained user preference prediction model.

[0012] Preferably, the process of debiasing the user embedding and the video embedding includes: training the embedding layer and calculating the pairwise sorting loss; adjusting the embedding layer parameters according to the pairwise sorting loss to obtain a trained embedding layer; and debiasing the user embedding and the video embedding output by the embedding layer after the training is completed according to the user embedding and the video embedding obtained during the training process to obtain a debiased user embedding and a debiased video embedding.

[0013] Furthermore, the formulas for debiasing user embedding and debiasing video embedding are obtained as follows:

[0014]

[0015] in, represents the debiased user embedding of video i, Q i represents the video embedding of video i, α 1 is the first control parameter, represents the average video embedding obtained by the embedding layer in multiple iterations, represents the debiased user embedding of user u, P u represents the user embedding of user u, α 2 is the second control parameter, represents the average user embedding obtained by the embedding layer over multiple iterations.

[0016] Preferably, the pairwise ranking loss is:

[0017]

[0018] Among them, L BPR represents the pairwise ranking loss, σ represents the sigmoid activation, represents the matching score between user u and positive sample video i, represents the matching score between user u and negative sample video j, λ Θ represents the learning rate, Θ represents the embedding layer parameter, and D s represents a triplet set, (u,i,j) represents the triplet of user u, positive sample video i and negative sample video j.

[0019] Preferably, the process of obtaining the user-specific feature importance weight and the video-specific feature importance weight includes:

[0020] The MLP-based gating network is used to process the user embedding and video embedding respectively to obtain the user gating weight and video gating weight;

[0021] Concatenate all debiased user embeddings of the user to obtain a concatenated debiased user embedding; concatenate all debiased video embeddings of the video to obtain a concatenated debiased video embedding;

[0022] The user-specific feature importance weights are calculated based on the user gating weights and the spliced ​​debiased user embedding; the video-specific feature importance weights are calculated based on the video gating weights and the spliced ​​debiased video embedding.

[0023] Furthermore, the formula for calculating the importance weight of user-specific features and video-specific features is expressed as:

[0024] h 1 =2σ(g 1 )⊙e

[0025] h 2 =2σ(g 2 )⊙f

[0026] Among them, h 1 represents the importance weight of user-specific features, g 1 represents the user gating weight, e represents the concatenated debiased user embedding, σ represents the sigmoid activation function, ⊙ represents the multiplication of the corresponding position elements, and h 2 represents the important weight of video-specific features, g 2 represents the video gating weight and f represents the concatenated debiased video embedding.

[0027] Preferably, the factor decomposition machine processes the user-specific feature importance weight and the video-specific feature importance weight as follows:

[0028]

[0029] z i =h 1,i +h 2,i

[0030] in, represents the user interest representation vector, θ 0 represents a constant, θ i is the linear network parameter, z i represents the i-th splicing feature, z j represents the jth concatenated feature, k represents the number of feature types, <θ i ,θ j > is the cross model parameter, h1,i The i-th feature representation representing the importance weight of the user-specific feature, h 2,i The i-th feature representation representing the importance weight of a specific feature of the video.

[0031] Preferably, the process of processing the user interest representation vector by the output layer includes: processing the user interest representation vector using a sigmoid function to obtain a user preference prediction result.

[0032] Preferably, the cross entropy loss is:

[0033]

[0034] Among them, loss represents the cross entropy loss, N represents the number of training samples, represents the predicted click probability of the jth sample, y j Represents the actual click label of the user for the jth sample.

[0035] The beneficial effects of the present invention are as follows: the present invention effectively increases the accuracy of the representation of users and video content by the embedded representation of categorical features and numerical features and the embedding of original data after debiasing, solves the problem that the recommendation model is too dependent on initial data, resulting in inaccurate predictions or single recommended content, and adopts stream-specific feature selection to expand the difference between the two streams, thereby generating differentiated feature inputs to select features, more efficiently utilizing the information of feature combinations, avoiding the problem of inefficiency or even ineffectiveness caused by the intersection of invalid features in the learning process of the neural network, and finally introducing a factor decomposition machine to fully learn the intersection of low-order and high-order features, thereby achieving accurate and effective user preference prediction through feature cross-combination, and improving the accuracy of user preference prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a structural diagram of the user preference prediction model in the present invention;

[0037] Figure 2 Schematic diagram of the processing of the feature screening layer in the user preference prediction model of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The present invention proposes a user preference prediction method based on data debiasing, such as Figure 1 As shown, the method includes: obtaining video data and user data; inputting the video data and user data into a trained user preference prediction model to obtain a user preference prediction result. The user preference prediction model includes an embedding layer, an interest capture layer, a feature screening layer, and an output layer.

[0040] The training process of the user preference prediction model includes:

[0041] S1: Obtain video data and user data for training; use an embedding layer to process the video data and user data to obtain user embedding and video embedding.

[0042] User data includes the user's age, activity level and occupation, and video data includes category, duration and author. The training label is whether the user interacts with the video (clicks), such as watching, commenting, downloading, etc.

[0043] The embedding layer is used to process video data and user data. Specifically:

[0044] The embedding layer is a neural network layer commonly used in deep learning. Its main function is to convert some low-dimensional or high-dimensional features into the same dimension to facilitate the subsequent unification of vector dimensions. Different types of features are divided and encoded and vectorized to obtain user embedding and video embedding, which are expressed as:

[0045] P m =[α 1 ,α 2 ,...,α i ,...,α M ]

[0046] Q n =[β 1 ,β 2 ,...,β j ,...,β N ]

[0047] Among them, M and N represent the number of all user features and video features; for category features, α i is a one-hot vector. For discrete features, α i is a scalar value. Similarly, when the jth feature of the video content is a sparse feature, β j One-hot vector, when it is a non-categorical feature, β j Is a scalar value. m is the embedding representation of the mth user, Q n is the embedded representation of the nth video content.

[0048] S2: Debiasing the user embedding and the video embedding to obtain debiased user embedding and debiased video embedding.

[0049] Train the embedding layer, specifically:

[0050] Pair users with video content, and divide samples into positive samples and negative samples based on whether the users interact with the video content. That is, when the user interacts with the video, including watching, commenting, downloading, etc., the sample pair is considered a positive sample, and when there is no interaction, the sample pair is considered a negative sample. This distinction is made because, for gradient updates, the model's gradient weight for positive samples is greater than that for negative samples, so the update direction of the combined gradient is more inclined to the positive gradient direction, which also leads to a deviation in the gradient direction learned by the model.

[0051] In order to reduce the impact of positive gradients on the direction of the gradient, the paired sorting loss is used to measure the difference between positive and negative samples, and the vector representation parameters of the user and video content are updated. The paired sorting loss is calculated as follows:

[0052]

[0053] Among them, L BPR represents the pairwise ranking loss, σ represents the sigmoid activation, σ(x) = 1 / (1+exp(-x)); represents the matching score between user u and positive sample video i, represents the matching score between user u and negative sample video j; Θ represents the learning rate, which is used to control the speed of adjustment of the embedding layer parameters; Θ represents the embedding layer parameters, D s represents a triplet set, (u,i,j) represents the triplet of user u, positive sample video i and negative sample video j.

[0054] By adjusting the embedding layer parameters using pairwise sorting loss, a trained embedding layer can be obtained.

[0055] For the embedding layer parameter Θ, the gradient of the BPR loss can be written as:

[0056]

[0057] When updating the gradient of the embedding vector:

[0058]

[0059] Then in the tth iteration, the forward video embedding Negative video embedding and user embed The potential vector of is updated as follows:

[0060]

[0061] Among them, lr is the learning rate.

[0062] Video embedding and user embedding can be split as follows:

[0063]

[0064] in, Used to capture the impact of video duration, Used to capture the true characteristics of the content, Captures user activity. Indicates the consistency of the user with other mainstream users, Represents the user's real interest. Since active users tend to have more observed interactions, it is in the form of a product.

[0065] Since active users will have more observed interactions with video content, their accumulated gradients based on these video content may be larger than those of other less active users. For example, if user u 1 and u 2 Preferences for video content are similar, but u 1 Than u 2 Active, then u 1 and u 2 The update direction of the embedding vector of will be similar, u 1 The vector norm of will be much larger, so such a large embedding will increase the gradient update of its corresponding project, exacerbating the imbalance problem between positive and negative gradients. In order to reduce the bias propagation of highly active users, each user needs to be embedded into a unit vector during the training phase:

[0066]

[0067] In order to mitigate the impact of the video length itself, for a long popular video project, its positive interaction with the user is often observed in the training samples, causing the learned embedding to shift in the direction of positive interaction. In order to correct this shift, the overall gradient is corrected by correcting the accumulated positive gradient, which can achieve debiasing of the user embedding and video embedding. Therefore, in the debiasing calculation layer, the user embedding and video embedding output by the embedding layer after training can be debiased according to the user embedding and video embedding obtained during the training process. The formula is as follows:

[0068]

[0069]

[0070] in, represents the debiased user embedding of video i, Qi represents the video embedding of video i, α 1 represents a first control parameter, which is a scalar used to control the projection size of the initial video embedding in the direction of the average video embedding; represents the average video embedding obtained by the embedding layer during multiple iterations of the training process, represents the debiased user embedding of user u, P u represents the user embedding of user u, α 2 represents the second control parameter, which is a scalar used to control the projection size of the initial user embedding in the direction of the average user embedding; Represents the average user embedding obtained by the embedding layer during multiple iterations of the training process.

[0071] S3: The user embedding, video embedding, debiased user embedding and debiased video embedding are processed using a feature filtering layer to obtain user-specific feature importance weights and video-specific feature importance weights.

[0072] The feature selection layer expands the difference between the two streams through stream-specific feature selection, thereby generating differentiated feature inputs. A stream-specific feature module is used to reweight the feature inputs and perform feature gating from different views to generate user-specific feature importance weights and video-specific feature importance weights, respectively. Specifically, Figure 2 As shown:

[0073] The MLP-based gating network is used to process the user embedding and video embedding respectively to obtain the user gating weight and video gating weight:

[0074] g 1 =Gate 1 (x 1 ),g 2 =Gate 2 (x 2 )

[0075] Among them, Gate i represents an MLP-based gating network that transforms the flow-specific conditional features x i As input, it outputs the element-wise gating weight g i . x can be flexibly selected from a set of user / video features i , x 1 For user embedding, x 2 Embed the video; or i Set as learnable parameters.

[0076] Concatenate all debiased user embeddings of a user Get the spliced ​​debiased user embedding e; all debiased video embeddings of the spliced ​​video Get the spliced ​​and debiased video embedding f.

[0077] The user-specific feature importance weights are calculated based on the user gating weights and the spliced ​​debiased user embedding; the video-specific feature importance weights are calculated based on the video gating weights and the spliced ​​debiased video embedding; the calculation formula is:

[0078] h 1 =2σ(g 1 )⊙e

[0079] h 2 =2σ(g 2 )⊙f

[0080] Among them, h 1 represents the importance weight of user-specific features, g 1 represents the user gating weight, e represents the concatenated debiased user embedding, σ represents the sigmoid activation function, and the feature importance weight is converted to the range of [0,2] by using the sigmoid function σ and multiplying it by 2, with an average value of 1; ⊙ represents the multiplication of the corresponding position elements, h 2 represents the important weight of video-specific features, g 2 represents the video gating weight and f represents the concatenated debiased video embedding.

[0081] S4: A factor decomposition machine is used to process the user-specific feature importance weights and the video-specific feature importance weights to obtain a user interest representation vector.

[0082] The interest capture layer uses a factorization machine, which is suitable for estimating interactions between variables in sparse scenarios. The factorization machine destroys the independence of interaction parameters by decomposing the interaction parameters. Usually, this means that the data of one interaction also helps to estimate the parameters of related interactions. The factorization machine for the second-order interaction is calculated as follows:

[0083]

[0084] z i =h 1,i +h 2,i

[0085] in, represents the user interest representation vector, θ 0 represents a constant, θ i is the linear network parameter used to learn the first-order features, z i represents the i-th splicing feature, z j represents the jth concatenated feature, k represents the number of feature types, <θ i ,θ j > is the cross model parameter used to learn the second-order feature interaction, h 1,i The i-th feature representation representing the importance weight of the user-specific feature, h2,i The i-th feature representation representing the importance weight of a specific feature of the video.

[0086] S5: Input the user interest representation vector into the output layer for processing to obtain the user preference prediction result.

[0087] The output layer uses the sigmoid function to process the user interest representation vector to obtain the user preference prediction result, that is, the probability of the user clicking on the video content.

[0088] S6: Calculate the model cross entropy loss and adjust the model parameters according to the model cross entropy loss to obtain a trained user preference prediction model.

[0089] Back propagation is performed through cross entropy loss to optimize the model parameters. When the model loss converges or reaches the maximum number of iterations, the training is stopped and the model parameters are kept to obtain a trained user preference prediction model.

[0090] Among them, the cross entropy loss is:

[0091]

[0092] Among them, loss represents the cross entropy loss, N represents the number of training samples, represents the predicted click probability of the jth sample, y j Represents the actual click label of the user for the jth sample.

[0093] After the model training is completed, video data and user data are obtained, and the video data and user data are input into the trained user preference prediction model to obtain the user preference prediction results; videos are recommended to users based on the user preference prediction results to achieve accurate recommendations for users.

[0094] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A user preference prediction method based on data debiasing, characterized in that: include: Get video data and user data; Input the video data and user data into the trained user preference prediction model to obtain the user preference prediction result; The training process of the user preference prediction model includes: S1: Obtain video data and user data for training; use an embedding layer to process the video data and user data to obtain user embedding and video embedding; S2: Debias the user embedding and the video embedding to obtain debiased user embedding and debiased video embedding; S3: The user embedding, the video embedding, the debiased user embedding and the debiased video embedding are processed using a feature filtering layer to obtain user-specific feature importance weights and video-specific feature importance weights; S4: Use a factorization machine to process the user-specific feature importance weights and the video-specific feature importance weights to obtain a user interest representation vector; S5: Input the user interest representation vector into the output layer for processing to obtain the user preference prediction result; S6: Calculate the model cross entropy loss and adjust the model parameters according to the model cross entropy loss to obtain a trained user preference prediction model.

2. The method for predicting user preferences based on data debiasing according to claim 1, characterized in that: The process of debiasing the user embedding and the video embedding includes: training the embedding layer and calculating the pairwise sorting loss; adjusting the embedding layer parameters according to the pairwise sorting loss to obtain the trained embedding layer; debiasing the user embedding and the video embedding output by the embedding layer after the training is completed according to the user embedding and the video embedding obtained during the training process to obtain the debiased user embedding and the debiased video embedding.

3. The method for predicting user preferences based on data debiasing according to claim 2, characterized in that: The formulas for getting the debiased user embedding and debiased video embedding are: in, represents the debiased user embedding of video i, Q i represents the video embedding of video i, α1 is the first control parameter, represents the average video embedding obtained by the embedding layer in multiple iterations, represents the debiased user embedding of user u, P u represents the user embedding of user u, α2 is the second control parameter, represents the average user embedding obtained by the embedding layer over multiple iterations.

4. The method for predicting user preferences based on data debiasing according to claim 1, characterized in that: The pairwise ranking loss is: Among them, L BPR represents the pairwise ranking loss, σ represents the sigmoid activation, represents the matching score between user u and positive sample video i, represents the matching score between user u and negative sample video j, λ Θ represents the learning rate, Θ represents the embedding layer parameter, and D s represents a triplet set, (u,i,j) represents the triplet of user u, positive sample video i and negative sample video j.

5. The method for predicting user preferences based on data debiasing according to claim 1, characterized in that: The process of obtaining the user-specific feature importance weight and the video-specific feature importance weight includes: The MLP-based gating network is used to process the user embedding and video embedding respectively to obtain the user gating weight and video gating weight; Concatenate all debiased user embeddings of the user to obtain a concatenated debiased user embedding; concatenate all debiased video embeddings of the video to obtain a concatenated debiased video embedding; The user-specific feature importance weights are calculated based on the user gating weights and the spliced ​​debiased user embedding; the video-specific feature importance weights are calculated based on the video gating weights and the spliced ​​debiased video embedding.

6. The method for predicting user preferences based on data debiasing according to claim 5, characterized in that: The formula for calculating the importance weight of user-specific features and video-specific features is expressed as: h1=2σ(g1)⊙e h2=2σ(g2)⊙f Among them, h1 represents the user-specific feature importance weight, g1 represents the user gating weight, e represents the concatenated debiased user embedding, σ represents the sigmoid activation function, ⊙ represents the multiplication of elements at corresponding positions, h2 represents the video-specific feature importance weight, g2 represents the video gating weight, and f represents the concatenated debiased video embedding.

7. The method for predicting user preferences based on data debiasing according to claim 1, characterized in that: The formula for the factorization machine to process the user-specific feature importance weights and video-specific feature importance weights is: z i =h 1,i +h 2,i in, represents the user interest representation vector, θ0 represents a constant, θ i is the linear network parameter, z i represents the i-th splicing feature, z j represents the jth concatenated feature, k represents the number of feature types, <θ i ,θ j > is the cross model parameter, h 1,i The i-th feature representation representing the importance weight of the user-specific feature, h 2,i The i-th feature representation representing the importance weight of a specific feature of the video.

8. The method for predicting user preferences based on data debiasing according to claim 1, characterized in that: The processing process of the output layer on the user interest representation vector includes: using the sigmoid function to process the user interest representation vector to obtain the user preference prediction result.

9. The method for predicting user preferences based on data debiasing according to claim 1, characterized in that: The cross entropy loss is: Among them, loss represents the cross entropy loss, N represents the number of training samples, represents the predicted click probability of the jth sample, y j Represents the actual click label of the user for the jth sample.

Citation Information

Patent Citations

  • Heterogeneous graph neural network-based recommendation method

    CN112990972A

  • E-commerce platform session awareness recommendation prediction method based on long and short term interests

    CN115293812A

  • Determination method of prediction model and related device

    CN117874321A

  • Training method and system of neural network model and prediction method and system

    US20210264272A1