Preference prediction method, electronic device, and storage medium

Abstract

The application provides a preference prediction method, an electronic device and a storage medium. The method comprises the following steps: determining a first object set of user historical interactions, a second object set affected by popularity bias and a third object set affected by amplification subjective bias; encoding the first object set, the second object set and the third object set into a first one-hot vector, a second one-hot vector and a third one-hot vector respectively; inputting the first one-hot vector, the second one-hot vector and the third one-hot vector into a variational autoencoder established in advance respectively to obtain a first latent semantic vector, a second latent semantic vector and a third latent semantic vector correspondingly; calculating a debiased latent semantic vector according to the first latent semantic vector, the second latent semantic vector and the third latent semantic vector; and inputting the debiased latent semantic vector into a decoder established in advance to predict an object preferred by a user. The application can effectively improve the accuracy of user preference prediction.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computers, and particularly relates to a preference prediction method, an electronic device and a storage medium. BACKGROUND

[0002] A recommendation system can predict user preferences, that is, help users find goods or information that meet their personal preferences from a large amount of data, and is widely used in online applications such as e-commerce, social networks and advertising. However, the recommendation system has been affected by various biases, such as popularity bias and amplification of subjective bias.

[0003] In order to alleviate these two types of bias, many debiasing methods have been proposed in recent years. For solving popularity bias, the main method is to reweight items, including strengthening the weight of unpopular items and reducing the weight of popular items. For amplification of subjective bias, there are three main methods in the academic field, which solve the problem of amplification of subjective bias from fairness, diversity and standardization. These three methods essentially adjust the distribution of different types of goods in training. With the emphasis on causal reasoning methods in machine learning in recent years, causal methods have also been widely used in the debiasing field of recommendation systems. The academic field has also given methods to solve popularity bias and amplification of subjective bias from the perspective of causal influence.

[0004] However, the above-mentioned debiasing methods still have some problems and defects. First of all, all the above-mentioned methods reduce the mixed factors that may cause bias by limiting or adjusting the weight of goods, but this adjustment is blind and has no explicit supervision signal. Secondly, all these debiasing methods are for a single specific bias, either popularity bias or amplification of subjective bias, ignoring the complex rectification relationship between the two biases. For example, there can be two reasons for a user to click on an item, the item is popular or the item meets his historical preferences. If the popularity bias is adjusted too much, the matching degree is likely to be the primary reason for the user to interact with the item, which can lead to more serious amplification of subjective bias. Conversely, the over-restricted project meets the majority of the user's preferences and can lead to popular items dominating the recommendation list. Thirdly, data sparsity makes debiasing more difficult. If a user has a sufficient number of interaction projects in his historical profile, the model has enough evidence to analyze his preferences and choose which projects are affected by bias. However, if the data is sparse, it becomes difficult to identify the mainstream preferences of the user and the bias. Therefore, the accuracy of the existing recommendation system for predicting user preferences is low. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a preference prediction method, an electronic device and a storage medium, which can improve the accuracy of user preference prediction.

[0006] The content of the present application comprises a preference prediction method, comprising:

[0007] determining a first object set of user historical interactions, a second object set affected by popularity bias, and a third object set affected by amplification subjective bias;

[0008] corresponding encoding the first object set, the second object set and the third object set into a first one-hot vector, a second one-hot vector and a third one-hot vector respectively;

[0009] inputting the first one-hot vector, the second one-hot vector and the third one-hot vector into a pre-established variational autoencoder respectively, and correspondingly obtaining a first latent semantic vector, a second latent semantic vector and a third latent semantic vector;

[0010] calculating a debiased latent semantic vector according to the first latent semantic vector, the second latent semantic vector and the third latent semantic vector;

[0011] inputting the debiased latent semantic vector into a pre-established decoder to predict the object preferred by the user.

[0012] Optionally, the determining a first object set of user historical interactions, a second object set affected by popularity bias, and a third object set affected by amplification subjective bias comprises:

[0013] obtaining a first object set of user historical interactions;

[0014] calculating a popularity degree score and a matching degree score of each object in the first object set;

[0015] adding an object in the first object set with a popularity degree score higher than a preset first popularity degree value and a matching degree score lower than a preset first matching degree value to the second object set;

[0016] adding an object in the first object set with a popularity degree score lower than a preset second popularity degree value and a matching degree score higher than a preset second matching degree value to the third object set.

[0017] Optionally, the calculation formula of the popularity degree score is:

[0018]

[0019] wherein, S p (u, i) is the popularity degree score of object i for user u, num(i) is the number of times object i is interacted by all users, and X includes all objects interacted by user u;

[0020] The calculation formula of the matching degree score is:

[0021]

[0022] d u =∑ j∈X d j ;

[0023] wherein S m (u,i) is a matching degree score for object i of user u, d i is an attribute vector of object i, d u is an attribute vector of user u.

[0024] Optionally, the first one-hot vector, the second one-hot vector and the third one-hot vector are respectively input into a pre-established variational autoencoder, to correspondingly obtain a first latent semantic vector, a second latent semantic vector and a third latent semantic vector, including:

[0025] the first one-hot vector, the second one-hot vector and the third one-hot vector are respectively input into a pre-established decoupled variational autoencoder, to correspondingly obtain a first posterior distribution, a second posterior distribution and a third posterior distribution;

[0026] a vector is randomly sampled from a standard normal distribution;

[0027] the first latent semantic vector is generated according to the first posterior distribution and the sampling vector;

[0028] the second latent semantic vector is generated according to the second posterior distribution and the sampling vector;

[0029] the third latent semantic vector is generated according to the third posterior distribution and the sampling vector.

[0030] Optionally, a generation formula of the first latent semantic vector is:

[0031] z x =μ x +∈⊙σ x ;

[0032] wherein z x is the first latent semantic vector, μ x is a mean vector in the first posterior distribution, σ x is a variance vector in the first posterior distribution, and ∈ is the sampling vector;

[0033] a generation formula of the second latent semantic vector is:

[0034] z m =μ m +∈⊙σ m ;

[0035] wherein z m is the second latent semantic vector, μ m is the mean vector in the second posterior distribution, σ m is the variance vector in the second posterior distribution;

[0036] The generation formula of the third latent semantic vector is:

[0037] z p = μ p + ∈ ⊙ σ p ;

[0038] wherein z p is the third latent semantic vector, μ p is the mean vector in the third posterior distribution, σ p is the variance vector in the third posterior distribution;

[0039] The calculation formula of the debiased latent semantic vector is:

[0040] z = μ x - μ m - μ p + ∈ ⊙ (σ x - σ m - σ p ).

[0041] Optionally, the method further comprises:

[0042] inputting the second latent semantic vector into the decoder to predict the preference object of the user affected by the popularity bias;

[0043] inputting the third latent semantic vector into the decoder to predict the preference object of the user affected by the amplified subjective bias.

[0044] Optionally, the method further comprises:

[0045] for each object, predicting a first preference score of the user not affected by the popularity bias and not affected by the amplified subjective bias, a second preference score of the user not affected by the popularity bias, and a third preference score of the user not affected by the amplified subjective bias;

[0046] adding N1 objects with the highest first preference scores to the first object set;

[0047] adding N2 objects with the highest second preference scores to the third object set;

[0048] adding N3 objects with the highest third preference scores to the second object set.

[0049] Optionally, the first preference score, the second preference score and the third preference score of the user, which are not affected by the popularity bias and the amplification subjective bias, are predicted for each object, comprising:

[0050] A structured model is constructed based on the matching degree exogenous variable and the popularity degree exogenous variable for each object.

[0051] The structured model is set to predict the first preference score of the user, which is not affected by the popularity bias and the amplification subjective bias, by setting the matching degree exogenous vector and the popularity degree exogenous variable to conform to a standard normal distribution.

[0052] The structured model is set to predict the second preference score of the user, which is not affected by the popularity bias, by setting the popularity degree exogenous variable to conform to a standard normal distribution and setting the matching degree exogenous vector to conform to a pre-trained posterior distribution.

[0053] The structured model is set to predict the third preference score of the user, which is not affected by the amplification subjective bias, by setting the matching degree exogenous vector to conform to a standard normal distribution and setting the popularity degree exogenous variable to conform to a pre-trained posterior distribution.

[0054] The application further provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned preference prediction method when executing the program.

[0055] The application further provides a non-transitory computer readable storage medium, which stores computer instructions for making the computer execute the above-mentioned preference prediction method.

[0056] The application has the beneficial effect that the first object set, the second object set and the third object set of the user historical interaction are determined, which are affected by the popularity bias and the amplification subjective bias; the first object set, the second object set and the third object set are respectively coded into a first one-hot vector, a second one-hot vector and a third one-hot vector; the first one-hot vector, the second one-hot vector and the third one-hot vector are respectively input into a pre-established decoupling variational autoencoder to obtain a first latent semantic vector, a second latent semantic vector and a third latent semantic vector; the decoupling latent semantic representation is decoupled to predict the object of the user preference, thereby effectively improving the accuracy of the user preference prediction. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0058] Figure 1 The flowchart of the preference prediction method provided by the embodiments of the present application.

[0059] Figure 2 The causal diagram of user interaction provided by the embodiments of the present application.

[0060] Figure 3 The framework diagram of the DB-VAE model in the preference prediction method provided by the embodiments of the present application.

[0061] Figure 4 The causal diagram of user interaction with exogenous variables provided by the embodiments of the present application.

[0062] Figures 5a to 5d The performance comparison diagram of different models in different sparsity groups.

[0063] Figures 6a to 6d The performance comparison diagram of different models in different debiasing thresholds.

[0064] Figure 7 The structural diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be described clearly and completely in the following description with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort belong to the scope of protection of the present application.

[0066] The recommendation system usually faces various kinds of biases. There is a popularity bias in the recommendation system, and most of the objects in the data set of the recommendation system are unpopular objects (the number of interactions with the user is less than 50 times), but the frequency of being predicted as the user's preference is not high, and the popular objects, although the proportion is small, are often predicted as the user's preferred objects. There is also an amplification of subjective bias in the recommendation system. For example, in the interaction history of a certain user, the proportion of comedy movies is 70%, and the proportion of action movies is 30%. In the training process of the recommendation system, the proportion of the user's mainstream preference is often overestimated, such as the proportion of comedy movies is amplified to 90%, and the proportion of action movies is reduced to 10%, which causes the preference of comedy movies to be exaggeratedly amplified in prediction, and in the continuous feedback loop, the prediction of the user's preference is more and more limited to comedy movies, causing problems such as "information echo".

[0067] To solve the above problems, the embodiment of the application provides a preference prediction method. As shown in the figure, Figure 1 The preference prediction method provided by the embodiment of the application comprises steps 101 to 105, and the specific steps are as follows:

[0068] Step 101, determining a first object set of user historical interactions, a second object set affected by a popularity bias, and a third object set affected by an amplification of subjective bias.

[0069] The first object set is a set of all objects that the user has interacted with, the second object set is a set of all objects that may be affected by the popularity bias in the user's historical interactions, and the third object set is a set of all objects that may be affected by the amplification of subjective bias in the user's historical interactions. Wherein, the object can be any object that the user can click, such as a movie, an item, etc.

[0070] The second object set and the third object set can be screened from the first object set. As shown in the figure, Figure 2 In the causal diagram of user interaction, the two direct reasons for causing the user's click behavior (C) are the popularity of the object (B) and the matching degree of the user and the object (M). As can be seen from Figure 2 The three nodes form a collision, and C is the collision of M and B. Then according to the causal theory, M and B, which seem to be irrelevant, are related when C is taken as a condition. For example, if a user clicks an object that is not very matched with his historical preference, it is very likely that the object is very popular. Similarly, if the user clicks an object that is not very popular, it is very likely that the object is very matched with the user's historical preference. This phenomenon is called collision effect. According to the collision effect, objects that may cause popularity bias and amplification of subjective bias can be selected.

[0071] Specifically, the determining the first object set, the second object set affected by the popularity bias and the third object set affected by the amplification subjective bias in the step 101 comprises:

[0072] obtaining a first object set of user historical interactions;

[0073] calculating a popularity score and a matching score of each object in the first object set;

[0074] adding an object in the first object set with a popularity score higher than a preset first popularity value and a matching score lower than a preset first matching value to the second object set;

[0075] adding an object in the first object set with a popularity score lower than a preset second popularity value and a matching score higher than a preset second matching value to the third object set.

[0076] By collecting user historical data (the collection of user historical data has been authorized by the user), a first object set of user historical interactions is obtained, for example, the first object set can include all the movies watched by the user. In order to facilitate the specific comparison of the difference between the popularity and the matching degree, the popularity score of each object in the first object set and the matching score with the user are defined respectively, as follows:

[0077]

[0078]

[0079] wherein, S p (u, i) is the popularity score of the object i for the user u, num(i) is the number of times the object i is interacted by all users, and X includes all the objects interacted by the user u. The popularity of the object is compared among the objects interacted by the same user, so num(i) is divided by the sum of the popularity of other objects of the current user u as the popularity score S p (u, i) of the object i in the interaction of the current user u.

[0080] S m (u, i) is the matching score of the object i for the user u, d i is the attribute vector of the object i, and d uLet u be the attribute vector of user u. If all movies in the movie dataset have five attribute labels: "comedy", "action", "crime", "adventure" and "horror", then the attribute vector is 5-dimensional. If a movie "Spider-Man" has two labels, "comedy" and "action", then its attribute vector is [1, 1, 0, 0, 0]. The user's attribute vector is obtained by summing the attribute vectors of all the movies that the user has interacted with.

[0081] Then, based on the calculated popularity and match scores, we can obtain two extreme sets of data that may contain biases. The first set consists of objects that are very popular but do not align well with users' historical interaction preferences; that is, objects with very high popularity scores but very low match scores. These objects may be affected by popularity bias, and we filter them out as the second set of objects, X. p .like Figure 3 As shown, in one embodiment, the popularity score of each object in the first object set X is compared with a preset first popularity value, and the matching score of each object in the first object set X is compared with a preset first matching value. The set of all objects with popularity scores higher than the first popularity value and matching scores lower than the first matching value is used as the second object set X. p In another implementation, all objects in the first object set X are sorted from highest to lowest popularity score, and simultaneously, all objects in the first object set X are sorted from highest to lowest matching score. Then, the second object set X... p for:

[0082] X p ={i|rank(S p (u, i))<k|X|,rank(S) m (u, i))>k|X|,i∈X}.

[0083] Among them, rank(S) p (u, i)) represents the popularity score of object i, ranked among all objects in the first object set X. m (u, i)) represents the matching score of object i, ranked among all objects in the first object set X. k∈[0, 1] is the degree of debiasing; the smaller k is, the less bias is extracted from the user's historical data.

[0084] The second extreme type of object consists of those that closely match a user's historical preferences but are rarely clicked by other users (unpopular). These objects may be affected by amplified subjective bias. If many such objects exist in a user's historical data, it could confine the user to their own narrow world, resulting in an increasingly narrow range of objects for predicting user preferences. Similarly, objects affected by amplified subjective bias are filtered out and used as the third object set X. m .like Figure 3 As shown, in one embodiment, the popularity score of each object in the first object set X is compared with a preset second popularity value, and the matching score of each object in the first object set X is compared with a preset second matching value. The set of all objects whose popularity score is lower than the second popularity value and whose matching score is higher than the second matching value is used as the third object set X. m In another implementation, all objects in the first object set X are sorted from highest to lowest popularity score, and simultaneously, all objects in the first object set X are sorted from highest to lowest matching score. Then, the second object set X... m for:

[0085] X m ={i|rank(S p (u, i))>k|X|,rank(S) m (u, i))<k|X|,i∈X}.

[0086] Step 102: Encode the first object set, the second object set, and the third object set into a first one-hot vector, a second one-hot vector, and a third one-hot vector, respectively.

[0087] The dimensions of the first, second, and third one-hot vectors are equal to the number of all types of objects interacting with by all users. For example... Figure 3 As shown, if an object j is located in the first object set X of user u, then the j-th bit of the first one-hot vector is encoded as 1; otherwise, it is 0, thus encoding the first object set X into the first one-hot vector x. Similarly, the second object set X is encoded into the first one-hot vector x. p Encoded as the second one-hot vector x p , the third object set X m Encoded as a third one-hot vector x m .

[0088] Step 103: Input the first one-hot vector, the second one-hot vector, and the third one-hot vector into the pre-established variational autoencoder to obtain the first latent semantic vector, the second latent semantic vector, and the third latent semantic vector.

[0089] like Figure 3As shown, the application pre-constructs a decoupled variational autoencoder (DB-VAE) including a variational autoencoder and a decoder. The variational autoencoder (VAE) is a deep latent model, and the variational autoencoder can effectively learn the latent representation from real data. Unlike traditional autoencoders (AE), the variational autoencoder ensures the regularity of the latent semantic space by regularizing the latent semantic space during training. In this way, the variational autoencoder solves the overfitting problem of the traditional autoencoder. The reason for selecting the variational autoencoder as the basic model is that it can learn the standardized user implicit feature representation in the bias elimination process.

[0090] Specifically, the first one-hot vector, the second one-hot vector and the third one-hot vector are input into the pre-established variational autoencoder in step 103, and the first latent semantic vector, the second latent semantic vector and the third latent semantic vector are obtained correspondingly, including:

[0091] The first one-hot vector, the second one-hot vector and the third one-hot vector are input into the pre-established decoupled variational autoencoder, and the first posterior distribution information, the second posterior distribution information and the third posterior distribution information are obtained correspondingly;

[0092] Randomly sampling a vector from a standard normal distribution;

[0093] Generating the first latent semantic vector according to the first posterior distribution information and the sampling vector;

[0094] Generating the second latent semantic vector according to the second posterior distribution information and the sampling vector;

[0095] Generating the third latent semantic vector according to the third posterior distribution information and the sampling vector.

[0096] It should be noted that the first one-hot vector x, the second one-hot vector x p and the third one-hot vector x m are input into the variational autoencoder respectively, and the first latent semantic vector z x , the second latent semantic vector z p and the third latent semantic vector z m are generated correspondingly, that is:

[0097]

[0098]

[0099]

[0100] Where φ represents the parameters in the variational autoencoder, and here a five-layer neural network is chosen as the variational autoencoder, z p This can be viewed as a user representation influenced by popularity bias, z m This can be viewed as a user representation influenced by amplified subjective bias. To reconstruct the user's representation unaffected by bias, a debiased user representation can be obtained through subtraction, i.e.:

[0101] z = z x -z m -z P .

[0102] Unlike traditional autoencoders, variational autoencoders assume that the latent semantic representation follows a prior standard distribution from which any sample can be decoded into a true data distribution. However, the random sampling operation is non-differentiable during training, which can prevent the model from training. The solution to this problem is to have the decoder generate a posterior distribution N(μ, σ²) and then use KL divergence to make it approximate the standard normal distribution as closely as possible. In fact, the output posterior distribution information of the variational autoencoder, i.e., the mean vector μ and variance vector σ of the posterior distribution N(μ, σ²), is as follows: (e.g., the first posterior distribution is...)

[0103]

[0104] Specifically, the mean vector μ x and variance vector σ x Generated by variational autoencoder:

[0105] μ x =g1(f φ (x));

[0106] logσ x 2 =g2(f φ (x)).

[0107] Among them, f φ () is a five-layer multilayer neural network in a variational autoencoder, and g1() and g2() are two different fully connected layers in the variational autoencoder. The second posterior distribution Pφ(z) m |x m ) and the third posterior distribution Pφ(z) p |x p It can also be obtained in the same way; the variational autoencoder obtains it from the first posterior distribution through sampling. Second posterior distribution and the third posterior distribution The latent semantic representation, i.e., the first latent semantic vector z, is obtained. xThe second latent semantic vector zm and the third latent semantic vector z P However, the sampling operation is not differentiable, so it is performed in the following way:

[0108] z x =μ x +∈⊙σ x ;

[0109] z m =μ m +∈⊙σ m ;

[0110] z p =μ p +∈⊙σ p .

[0111] Where, μ x Let σ be the mean vector in the first posterior distribution. x Let μ be the variance vector in the first posterior distribution. m Let σ be the mean vector in the second posterior distribution. m Let μ be the variance vector in the second posterior distribution. p Let σ be the mean vector in the third posterior distribution. p Let be the variance vector in the third posterior distribution. ∈ is a sampling vector, i.e., a vector from the standard normal distribution. The vector is randomly sampled from the vector, and ⊙ represents the vector product operation.

[0112] Step 104: Calculate the biased latent semantic vector based on the first latent semantic vector, the second latent semantic vector, and the third latent semantic vector.

[0113] The above method transforms sampling from N(μ, σ2) into sampling from... Mid-sampling ensures that gradient descent does not involve sampling operations, but is instead included in the sampling results, making the entire model trainable. Therefore, the user's debiased latent semantic vector z is represented as:

[0114] z = μ x -μ m -μ p +∈⊙(σ x -σ m -σ p ).

[0115] Step 105: Input the debiased latent semantic vector into a pre-established decoder to predict the object preferred by the user.

[0116] like Figure 3As shown, by inputting the debiased latent semantic vector z into a pre-built decoder, the user's preferred object can be predicted.

[0117]

[0118] Where θ represents the parameters of the decoder.

[0119] Furthermore, the method also includes:

[0120] The second latent semantic vector is input into the decoder to predict the user's preferred objects affected by popularity bias;

[0121] The third latent semantic vector is input into the decoder to predict the user's preferred objects that are affected by amplified subjective bias.

[0122] With user preference objects Similar to the prediction method, this invention can also predict user preferences affected by popularity bias. And the user's preferences influenced by amplified subjective biases. Right now:

[0123]

[0124]

[0125] It should be noted that the parameters φ in the variational autoencoder and θ in the decoder can be learned by optimizing the lower bound of evidence (ELBO):

[0126]

[0127]

[0128]

[0129]

[0130] in, Equivalent to the loss function of traditional methods, optimization can improve the model's predictions during training. To more closely approximate the true value x, the cross-entropy loss function is chosen here. KL(p φ (z x |x,x p x m )||p(z x ) represents the posterior distribution p φ (z x |x,x p x m ) and prior distribution p(z)x ), where p(z x ) ~ N(0, I), optimizing it can make the generated user latent semantic vector closer to the standard normal distribution. Similarly, the cross-entropy loss function is selected to optimize the optimization objectives of the above two (the optimization objective affected by the popularity bias and the optimization objective affected by the amplified subjective bias), so as to realize the overall optimization of the variational autoencoder and the decoder. m and ω p respectively represent the weight parameters of the optimization objectives affected by the popularity bias and the amplified subjective bias, and in some embodiments, ω m and ω p are both set to 0.5, which can achieve the best effect.

[0131] Further, the method further comprises:

[0132] For each object, predicting a first preference score of the user without being affected by the popularity bias and without being affected by the amplified subjective bias, a second preference score without being affected by the popularity bias, and a third preference score without being affected by the amplified subjective bias;

[0133] Adding N1 objects with the highest first preference score to the first object set;

[0134] Adding N2 objects with the highest second preference score to the third object set;

[0135] Adding N3 objects with the highest third preference score to the second object set.

[0136] Specifically, the method of predicting, for each object, a first preference score of the user without being affected by the popularity bias and without being affected by the amplified subjective bias, a second preference score without being affected by the popularity bias, and a third preference score without being affected by the amplified subjective bias, comprises:

[0137] For each object, constructing a structured model based on a matching degree exogenous variable and a popularity degree exogenous variable;

[0138] Setting the matching degree exogenous vector and the popularity degree exogenous variable to conform to a standard normal distribution, so that the structured model predicts a first preference score of the user without being affected by the popularity bias and without being affected by the amplified subjective bias;

[0139] Setting the popularity degree exogenous variable to conform to a standard normal distribution, and setting the matching degree exogenous vector to conform to a posterior distribution, so that the structured model predicts a second preference score of the user without being affected by the popularity bias;

[0140] The matching degree exogenous vector is set to comply with a standard normal distribution, and the popularity degree exogenous variable is set to comply with a posterior distribution, so that the structured model predicts a third preference score of the user without the influence of amplification subjective bias.

[0141] It should be noted that in order to solve the problem of data sparsity, Pearl's causal inference framework is adopted, which includes three steps, namely induction, intervention, and prediction. According to the causal graph Figure 2 , a basic model for predicting user preferences is constructed, and it is necessary to assume two exogenous variables α and β to ensure that the counterfactual prediction process can proceed smoothly, and the causal graph becomes as shown in FIG. 1. Figure 4

[0142] α is the exogenous variable of the matching degree M, that is, the matching degree exogenous variable, and β is the exogenous variable of the popularity degree B, that is, the popularity degree exogenous variable, indicating that the matching degree and the popularity degree are not a fixed value, but can also be affected by some uncertain random factors, such as the user sometimes prefers the popular object, but sometimes prefers the object matching his previous object, which is not fixed.

[0143] The structured model F is defined in the form of probability, and the matching degree exogenous variable α and the popularity degree exogenous variable β can also be regarded as noise. With the structured model F, the key "induction" step of causal inference can be performed. Before induction, a specific definition of F is needed, that is:

[0144]

[0145]

[0146]

[0147] Wherein, C(u, i) is the preference score of the user u to the object i, u∈U, U represents all users, i∈I, I represents all objects, E u is the initial word embedding vector of the user u, E i is the initial word embedding vector of the object i, α i is the matching degree exogenous variable of the object i, β i is the popularity degree exogenous variable of the object i, and are weight parameters of the object i.

[0148] According to the existing training set The structured model F can be obtained by optimizing the following cross-entropy loss function:

[0149]

[0150] ​where x(u, i) is the true value in the training set, indicating whether user u has interacted with item i or not. By optimizing the global cross-entropy loss function, a specific F is learned. During the process of learning the parameters of F, it is assumed that a and β are subject to a standard normal distribution However, the distribution of a and β is different for different data sets, and in fact the distribution of a and β is what is needed to be learned in the next step of the "induction" process.

[0151] In the "induction" process, the main goal is to estimate the posterior distribution of a and β from the training set O. Take a as an example, its posterior distribution can be obtained by the following Bayes rule:

[0152] p(a|O)∝p(a, O) = p(a)p(O|a).

[0153] However, the expression of the prior distribution p(a) is not known, and it is also very difficult to sample from the above formula. Therefore, the present application solves the above distribution by variational inference. Specifically, it is first assumed that a is subject to the posterior distribution q φ (a) ~ N(μ, σ), where μ and σ are all learnable parameters. By optimizing the KL divergence between q φ (a) and p(a|O), μ and σ can be learned. In order to achieve this goal, the evidence lower bound to be optimized is:

[0154]

[0155] Similarly, the posterior distribution of B can also be learned in this way.

[0156] In the "intervention" process, the present application wants to obtain three counterfactual distributions, p C (C|M, do(B)), pC(C|do(M), B) and p C (C|do(M), do(B)). These three counterfactual distributions try to answer three counterfactual questions, 1) If the user is not affected by the popularity bias, which objects will he interact with? 2) If the user is not affected by the amplification subjective bias, which objects will he interact with? 3) If the user is not affected by the two biases, which objects will he interact with? Where the do(B) operation is to sample S (u, i) from p (u, i), that is, it is assumed that the popularity exogenous vector β of all objects is subject to a standard normal distribution, that is, it is assumed that the user is not affected by the popularity bias. Similarly, do(M) is to assume that the matching exogenous vector α of all objects is subject to a standard normal distribution, that is, it is assumed that the user is not affected by the amplification subjective bias.

[0157] In the "prediction" process, using the previously learned structured model F, through the do-operation method, the user's preference ratings for different objects under different do-operations can be predicted. Specifically, the exogenous vector of matching degree α is set to follow a standard normal distribution, and the exogenous vector of popularity β is set to follow a standard normal distribution. The C(u, i) output by the structured model F is the user u's preference rating for object i when it is not affected by popularity bias and amplified subjective bias, i.e., the first preference rating. The exogenous vector of matching degree α is set to follow a pre-trained posterior distribution, and the exogenous vector of popularity β is set to follow a standard normal distribution. The C(u, i) output by the structured model F is the user u's preference rating for object i when it is not affected by popularity bias, i.e., the second preference rating. The exogenous vector of matching degree α is set to follow a standard normal distribution, and the exogenous vector of popularity β is set to follow a pre-trained posterior distribution. The C(u, i) output by the structured model F is the user u's preference rating for object i when it is not affected by amplified subjective bias, i.e., the third preference rating.

[0158] After predicting the user's first preference rating for each object, all objects are arranged in descending order of their first preference ratings. The first N1 (N1≥1) objects constitute the first counterfactual dataset X. counter The first counterfactual dataset X counter Add to the first object set X to update the first object set X, i.e., X = X ∪ X counter After predicting the user's second preference rating for each object, all objects are sorted in descending order of their second preference ratings. The top N² (N²≥1) objects constitute the second counterfactual dataset. The second counterfactual dataset Add to the third object set X m To update the third object set X m ,Right now After predicting the user's third preference rating for each object, all objects are arranged in descending order of their third preference ratings. The top N3 (N3≥1) objects constitute the third counterfactual dataset. The third counterfactual dataset Add to the second object set Xp to update the second object set Xp, i.e. Among them, N1, N2 and N3 can be set to 100.

[0159] Using the updated first object set X and the updated second object set X p and the updated third object set X m The variational autoencoder and decoder described above can be retrained to address the data shortage problem and further improve the accuracy of user preference prediction.

[0160] To verify the effectiveness of the DB-VAE model in the present application, three real datasets are used for experiments, namely MovieLens dataset, AliShop-7C and Amazon-book. Among them, MovieLens is a widely used dataset collected from the Movielens website, including movie category information and user attribute information. MovieLens uses two different sizes of MovieLens datasets, namely ML-1M and ML-20M. AliShop-7C and Amazon-book are datasets derived from e-commerce platforms. Amazon-book records how users rate different books on e-commerce platforms. Each dataset used in the experiment contains sufficient item features, such as movie types and book categories. For each dataset, entries with less than 5 interactions are filtered out, and only users with at least 2 interactions are retained to ensure data quality. To evaluate the model's ability to construct user representations from explicit interactions, many users are waiting for validation and testing. 80% of the interactions in the test set are used to evaluate the remaining 20% of the scoring model. Table 1 lists all the information of the preprocessed datasets.

[0161]

[0162] Table 1

[0163] Three hidden layer models are taken for all encoders, which means f φ is a three-layer neural network. For symmetry, the decoder also contains three layers. The dimension is set to 200 in the encoder and the potential representation of any hidden layer to 600 in the decoder. The model is trained using the adam algorithm, with each training batch containing 500 users, and all learning decay rates are 0.01. The model parameters that perform best in NDCG@100 during training are retained to validate and evaluate the performance of the best model on the test set.

[0164] Two classic evaluation methods in recommendation systems are adopted: NDCG@K and Recall@K, which are widely used in top-K recommendation. Both methods compare the predicted values of the model with the real values of user interactions in the test set obtained, according to the predicted values given by the model, the K with the highest scores are selected to form the recommendation list R u The two indicators are defined as follows:

[0165]

[0166] Among them, I[] is an indicator function, which is 1 if the predicted object is in the user's real test set, otherwise it is 0.

[0167] The denominator is K, which is the minimum of the number of true interactions of users in the test set.

[0168]

[0169]

[0170] The definition of NDCG@K(u) is DCG@K(u) divided by its theoretical maximum. Generally, Recall@K is much larger than NDCG@K, so we choose Recall@20 and NDCG@100 as evaluation metrics here.

[0171] The performance of DB-VAE proposed in the application is compared with four baselines proposed in recent years. All the baselines can be divided into two groups. The first group is a well-designed VAE model without debiasing operation [1, 2]. The second group is a debiasing framework suitable for any model [3, 4], which aims to eliminate popular bias or amplify subjective bias.

[0172] 1) Mult-VAE first extends the variational autoencoder of implicit feedback in top-k RS, and by assuming that the user representation is subject to a multivariate normal distribution and introducing different regularization parameters for the learning goal, Mult-VAE achieves good results in top-k recommendation.

[0173] 2) A new composite prior distribution of latent representation, a better evidence lower bound KL divergence weight setting method and an alternating parameter updating method are proposed to improve the Mult-VAE algorithm.

[0174] 3) MACR is a model-agnostic framework designed to eliminate popularity bias in RS. They first analyze the causal relationship and design a multi-task learning method to answer the counterfactual question of what the ranking score would be if the model only used item attributes. In order to compare the performance difference of debiasing methods rather than the basic model, the basic model of the MACR framework is changed to VAE.

[0175] 4) DecRS provides an approximation operator for backdoor adjustment, and backdoor adjustment is an important method to eliminate the influence of obstacles in causal inference theory, aiming to eliminate amplified subjective bias. For the same reason as MACR, the basic model of DecRS is changed to VAE.

[0176] Table 2 shows the overall performance of the baseline models and the proposed model in terms of Recall@20 and NDCG@100. The bolded results represent the winner in that column. Generally, DB-VAE shows the most advanced performance in both metrics on all datasets.

[0177]

[0178] Table 2

[0179] In all cases, the performance improvement of the disentangled debiasing framework on the vae model is the most obvious. Compared with other baselines, the DB-VAE proposed in the application achieves the best performance on the two data sets, especially on the Alishop-7c data set, the DB-VAE improves more than 5% on two indicators compared with the best baseline model. Even the smallest improvement can reach 2.4%, which is achieved on the ML-20M data set with Recall@20. The experimental results show that the disentangled debiasing method proposed in the application not only outperforms the cleverly designed VAE debiasing method, but also outperforms the single debiasing method.

[0180] Overall, the performance of the models using debiasing methods (MACR, DecRS and DB-VAE) is better than that of the models without considering debiasing (multi-vae and RecVAE), which shows that bias is ubiquitous in all data sets of the recommendation system, and debiasing methods are very important to improve the recommendation performance. Among the models without using debiasing methods, the performance of RecVAE is significantly better than that of multi-vae, which shows that a well-designed model will greatly improve the performance.

[0181] Regarding the difference in sparsity of the data sets, the ML-1M data is denser than the ML-20M data, but the performance of all models on the ML-1M is not as good as on the ML-20M. This may be because the ML-1M collects less data from Movielens, and these data may be incomplete information. Therefore, the correlation between objects and users cannot be fully reflected, resulting in bias in preference prediction. The ML-20M data set and Alishop-7c have the same sparsity, but the performance of all models on the two data sets differs greatly. As can be seen from the data set situation table, the number of users and interactions of the Alishop-7c data set is much smaller than that of the ML-20M, which leads to the lack of sufficient training samples in the Alishop-7c data set.

[0182] The application simultaneously eliminates two biases, in order to test the effect of the two debiasing components, two variants of DB-VAE are designed, each of which only debiases one bias. DB-VAE(P) only debiases the popularity bias, which means that only X and X p As the input of the model, the user representation also becomes z=z x -z pSimilarly, DB-VAE(M) only keeps the module that debiases the amplified subjective bias. In addition, the best baseline without debiasing, i.e., RecVAE, is designed to participate in the comparison, so that the improvement of different debiasing components can be intuitively seen. In order to better train the DB-VAE model, the model trained with counterfactual data is defined as DB-VAE(CD). By comparing the original DB-VAE with DB-VAE(CD), the proposed counterfactual data augmentation method can be evaluated, and the results are shown in Table 3.

[0183]

[0184] Table 3

[0185] In general, the disentangled debiasing method is more advantageous than the single debiasing framework with only one component, i.e., the original DB-VAE outperforms DB-VAE(P) and DB-VAE(M) on both indicators on all datasets, which shows that the disentangled debiasing method is correct and effective. In some extreme cases, the performance of the model with a single debiasing component is worse than the model without the debiasing component. For example, DB-VAE(M) is defeated by RecVAE on both the ML-1M dataset and the Alishop-7c dataset on Recall@20. This may be because eliminating only one bias in the prediction process can exacerbate the negative impact of the other bias on the recommendation. On ML-1M, DB-VAE(P) performs much better than DB-VAE(M), while on ML-20M and amazonbook, DB-VAE(M) performs much better. This phenomenon can indicate that in data-intensive cases, the recommendation system is greatly affected by the popularity bias, and eliminating the popularity bias can greatly improve the performance of the recommendation system. Conversely, on sparse datasets, the subjective bias may be more likely to be amplified, so debiasing operations based on subjective bias will be more beneficial to the improvement of the recommendation effect.

[0186] Counterfactual training data helps DB-VAE improve performance on all datasets except ML-1M, especially on amazonbook, where DB-VAE(CD) improves by 6% and 10% on Recall@20 and NDCG@100, respectively, compared to the original DB-VAE. This phenomenon may indicate that when the data is sparse, counterfactual data can help improve preference prediction, while when there is enough data to train the model, counterfactual data has the opposite effect. When the data is sparse, the model training will be insufficient due to the lack of data, and counterfactual data can be used to train the model well. When there is enough data to train the model well, the addition of counterfactual data may change the distribution of the original data, resulting in poor model performance.

[0187] To evaluate the performance of the model under different sparsity, the test users are evenly divided into 8 groups according to the sparsity (i.e., the number of interaction items). For simplicity, only the debiased models, MACR, DecRS, DB-VAE and DB-VAE(CD), are tested, and the results are shown in Figures 5a to 5d

[0188] From the 1st group to the 8th group, the performance of all models on all datasets shows an overall upward trend, which shows that the sparsity of the data makes it very difficult to improve the recommendation performance. On all datasets, DB-VAE(CD) always leads in the first group, especially on the amazonbook dataset, DB-VAE(CD) is nearly 40% higher than the original DB-VAE. This phenomenon also shows that in the case of data sparsity, counterfactual data indeed helps to improve the effectiveness of the model. On ML-20M and Alishop-7c, DB-VAE leads greatly in the last two groups, which may indicate that the present application is most effective when there is enough data to support model training.

[0189] The debiasing threshold k controls the bias extracted from the user's original data, the smaller k, the less bias extracted, and vice versa. When evaluating the impact of K on the performance of the DB-VAE model, for simplicity, only Recall@20 is used to represent the change in model performance, because the performance of NDCG@100 has a similar trend. In order to have an intuitive understanding of the debiasing effect, RecVAE is used as a comparison benchmark without considering the debiasing method. From Figures 6a to 6d It can be seen that the performance of the present application DB-VAE on all datasets shows a process of first rising and then falling with the increase of k. On the two movielens datasets, DB-VAE performs best when k is 0.5, while on Alishop-7c and amazonbook, DB-VAE performs best when This shows that different datasets have different degrees of bias, so it is necessary to find the appropriate degree of debiasing to make the model perform best on the dataset. When k is small, such as k = 0.1 and k = 0.2, although DB-VAE does not reach its best performance, the performance is only a little worse than RecVAE. But when k becomes larger after the best performance, the performance of DB-VAE will decrease significantly, and there will be a huge gap with the benchmark model RecVAE. This phenomenon shows that although slight debiasing cannot improve the performance of the model on the dataset, excessive debiasing will cause greater damage to the performance of the model. Therefore, appropriate debiasing is needed.

[0190] ​In summary, the application determines a first object set of user historical interactions, a second object set affected by popularity bias, and a third object set affected by enhanced subjective bias; encodes the first object set, the second object set, and the third object set into a first one-hot vector, a second one-hot vector, and a third one-hot vector, respectively; inputs the first one-hot vector, the second one-hot vector, and the third one-hot vector into a pre-established decoupling variational autoencoder, to obtain a first latent semantic vector, a second latent semantic vector, and a third latent semantic vector, respectively; calculates a debiased latent semantic vector according to the first latent semantic vector, the second latent semantic vector, and the third latent semantic vector; decouples the debiased latent semantic representation to predict an object preferred by the user, effectively improving the accuracy of user preference prediction. In addition, counterfactual data is generated to alleviate the data sparsity of the object set, to assist in training the decoupling variational autoencoder, and further improve the accuracy of user preference prediction.

[0191] Figure 7 A specific electronic device hardware structure schematic diagram provided by the embodiment is shown, which can include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 for internal communication.

[0192] The processor 1010 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present specification.

[0193] The memory 1020 can be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are saved in the memory 1020 and called and executed by the processor 1010.

[0194] The input / output interface 1030 is configured to connect an input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0195] The communication interface 1040 is configured to connect a communication module (not shown in the figure) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as a USB, a network cable, etc.) or a wireless manner (such as a mobile network, WIFI, Bluetooth, etc.).

[0196] The bus 1050 includes a channel to transmit information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0197] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain components necessary for the implementation of the embodiments of the present specification, and does not have to contain all the components shown in the figure.

[0198] The non-transitory computer-readable storage medium of the embodiment of the present application stores a plurality of instructions, which can be loaded by a processor to execute the steps in any of the preference prediction methods provided by the embodiments of the present application.

[0199] The non-transitory computer-readable medium of the embodiment includes permanent and non-permanent, removable and non-removable media, which can realize information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0200] Those of ordinary skill in the art will realize that the foregoing discussion of any of the embodiments has been presented for the purpose of illustration and description and is not intended to limit the scope of the disclosure (including the claims) to the examples set forth in the description or illustration of specific embodiments. Further, the steps of any of the methods disclosed herein do not have to be performed in the precise order described. The steps of various embodiments can be performed in any order, unless otherwise specified or required by the circumstances. Other variations and modifications of the embodiments disclosed herein, in addition to those described herein, will be apparent to those of ordinary skill in the art from the foregoing description and accompanying drawings. Such variations and modifications are intended to fall within the scope of the disclosure. Accordingly, the disclosure is not limited to that precisely as shown and described.

[0201] In addition, to simplify the description and discussion, and so as not to obscure the application, well-known power / ground connections to integrated circuit (IC) chips and other components can or can not be shown in the provided figures. Furthermore, devices can be shown in block diagram form in order to avoid obscuring the application, and this also applies to similar block diagrams wherever they can be found in the present disclosure. In the description provided herein, numerous specific

[0202] Although the application has been described in conjunction with specific embodiments thereof, numerous alternatives, modifications, and variations will be readily apparent to those of ordinary skill in the art. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0203] Embodiments of the present application are intended to cover all such alternatives, modifications, and variations as falling within the scope of the appended claims. Accordingly, any omission, modification, equivalent replacement, improvement, etc. made in the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A preference prediction method characterized by, The method comprises: determining a first object set of user historical interactions, a second object set affected by popularity bias, and a third object set affected by amplification subjective bias; encoding the first object set, the second object set, and the third object set into a first one-hot vector, a second one-hot vector, and a third one-hot vector, respectively; inputting the first one-hot vector, the second one-hot vector, and the third one-hot vector into a pre-established variational autoencoder to obtain a first latent semantic vector, a second latent semantic vector, and a third latent semantic vector, respectively; calculating a debiased latent semantic vector according to the first latent semantic vector, the second latent semantic vector, and the third latent semantic vector; inputting the debiased latent semantic vector into a pre-established decoder to predict an object preferred by the user; The method comprises: obtaining a first object set of user historical interactions; calculating a popularity score and a matching score of each object in the first object set; adding an object in the first object set with a popularity score higher than a preset first popularity value and a matching score lower than a preset first matching value to the second object set; adding an object in the first object set with a popularity score lower than a preset second popularity value and a matching score higher than a preset second matching value to the third object set; The calculation formula of the popularity score is: ； wherein, is a popularity score for objects of the user , is a number of times objects were interacted with by all users, includes all objects that were interacted with by the user . The calculation formula of the matching score is: ； ； wherein, is a match score for a user to an object of the user, is an attribute vector of the object and is an attribute vector of the user .​ 2. The preference prediction method of claim 1, wherein The method comprises: inputting the first one-hot vector, the second one-hot vector, and the third one-hot vector into a pre-established decoupled variational autoencoder to obtain a first posterior distribution, a second posterior distribution, and a third posterior distribution, respectively; randomly sampling a vector from a standard normal distribution; generating the first latent semantic vector according to the first posterior distribution and the sampling vector; generating the second latent semantic vector according to the second posterior distribution and the sampling vector; generating the third latent semantic vector according to the third posterior distribution and the sampling vector.

3. The preference prediction method of claim 2, wherein, The generation formula of the first latent semantic vector is: ； wherein, is the first latent semantic vector, is a mean vector in the first posterior distribution, is a variance vector in the first posterior distribution, is the sampling vector; The generation formula of the second latent semantic vector is: ； wherein, is the second latent semantic vector, is the mean vector in the second posterior distribution, is the variance vector in the second posterior distribution; The generation formula of the third latent semantic vector is: ； wherein, is the third latent semantic vector, is the mean vector in the third posterior distribution, is the variance vector in the third posterior distribution; The calculation formula of the debiased latent semantic vector is: 。 4. The preference prediction method of claim 1, wherein The method further comprises: inputting the second latent semantic vector into the decoder to predict an object preferred by the user affected by popularity bias; inputting the third latent semantic vector into the decoder to predict an object preferred by the user affected by amplification subjective bias.

5. The preference prediction method of claim 1, wherein, The method further comprises: predict, for each object, a first preference score of a user without popularity bias and without amplification subjective bias, a second preference score without popularity bias, and a third preference score without amplification subjective bias; add N1 objects with the highest first preference scores to the first object set; add N2 objects with the highest second preference scores to the third object set; add N3 objects with the highest third preference scores to the second object set.

6. The preference prediction method of claim 5, wherein, The predicting, for each object, a first preference score of a user without popularity bias and without amplification subjective bias, a second preference score without popularity bias, and a third preference score without amplification subjective bias, comprises: constructing a structured model based on a matching degree exogenous variable and a popularity degree exogenous variable for each object; setting the matching degree exogenous vector and the popularity degree exogenous variable to conform to a standard normal distribution, so that the structured model predicts a first preference score of a user without popularity bias and without amplification subjective bias; setting the popularity degree exogenous variable to conform to a standard normal distribution, and setting the matching degree exogenous vector to conform to a pre-trained posterior distribution, so that the structured model predicts a second preference score of a user without popularity bias; setting the matching degree exogenous vector to conform to a standard normal distribution, and setting the popularity degree exogenous variable to conform to a pre-trained posterior distribution, so that the structured model predicts a third preference score of a user without amplification subjective bias.

7. An electronic device, characterized by comprising: A computer program product comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the preference prediction method according to any one of claims 1 to 6 when executing the program.

8. A non-transitory computer-readable storage medium, comprising: The non-transitory computer readable storage medium stores computer instructions for causing the computer to perform the preference prediction method according to any one of claims 1 to 6.

Images