A recommendation method based on fair representation learning

Abstract

The present application discloses a recommendation method based on fair representation learning, which belongs to the field of algorithm recommendation, including: obtaining a set of items with historical user interactions and a set of users with historical interactions with each item as user-item neighborhood information; pre-training the representation vectors of the user set and the item set using a neighborhood aggregation mechanism based on the obtained neighborhood information to obtain a user pre-trained representation vector and an item pre-trained representation vector; performing debiasing processing on the user pre-trained representation vector to obtain a debiased user representation vector; constructing a recommendation model, taking the debiased user representation vector and the item pre-trained representation vector as input, and training the recommendation model; performing rating prediction on the user-item based on the trained recommendation model; and screening a candidate recommendation list based on the rating prediction results. In view of the problem of user attribute bias in the recommendation system in the prior art, the present application reduces the bias of user attributes in the recommendation system.

Description

Technical Field

[0001] The present application relates to the field of algorithm recommendation, and more specifically, to a recommendation method based on fair representation learning. Background Art

[0002] With the advent of the information age, recommendation systems have been widely used in various platforms such as e-commerce, employment, and healthcare, helping people quickly obtain useful information from massive amounts of data. However, recommendation systems also have problems of unfairness and bias. There is a long-tail distribution problem in recommended items, with popular items being over-recommended and a large number of non-popular items being ignored; users may be influenced by sensitive attributes such as gender, age, and cultural background, forming biases when choosing and accepting recommended content. The life cycle of a recommendation system includes three key components: users, data, and models, which form a feedback loop. When deviations occur, they will gradually amplify in this loop, exacerbating the problems of unfairness and bias.

[0003] Methods for achieving fairness in machine learning can be divided into three categories: preprocessing, internal processing, and post-processing. Preprocessing processes the training data, but may not eliminate the unfairness in the test data; post-processing reorders the recommendation results to achieve fairness, but may lose personalization; internal processing incorporates fairness indicators into the objective function from the perspective of model training, which is an effective way to balance accuracy and fairness.

[0004] The Chinese patent application, application number CN202110411553.6, published on April 16, 2021, discloses a deep learning recommendation method and system based on comments and item descriptions, obtains data samples for deep learning training, and the data samples include user sets, item sets, item comment texts, and item description data; constructs a recommendation network model for obtaining user prediction and scoring data for items through deep learning, and trains the model by inputting all item description data related to the user, the comment text received by each item, and the item description data, and combining the degree of association of each item with the user; trains and optimizes the recommendation network model through data samples; and uses the optimized recommendation network model to obtain the predicted scores of users for items in the target area. However, when constructing the recommendation network model, this method only uses information such as item descriptions, comment texts, and user-item associations, but does not consider sensitive attributes of users (such as gender, age, etc.), lacks targeted denoising and debiasing mechanisms, and is difficult to eliminate the user attribute bias learned by the model during training. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] In response to the problem of user attribute bias in recommendation systems in the prior art, the present application provides a recommendation method based on fair representation learning, which reduces the bias of user attributes in the recommendation system by debiasing user attributes.

[0007] 2. Technical solution

[0008] The purpose of this application is achieved through the following technical solutions.

[0009] The present specification provides a recommendation method based on fair representation learning, including: obtaining user-item neighborhood information, including a set of items with historical user interactions and a set of users with historical interactions for each item. A neighborhood aggregation mechanism is used to pre-train the representation vectors of the user set and the item set to obtain a user pre-trained representation vector and an item pre-trained representation vector. Specifically, it includes: constructing an adjacency matrix A of user-item interaction, exponentially weighting the elements in the adjacency matrix A to obtain a weighted adjacency matrix A', and aggregating and updating the node representations according to the weighted adjacency matrix A' to obtain a user pre-trained representation vector and an item pre-trained representation vector. Debiasing the user pre-trained representation vector to obtain a debiased user representation vector. Specifically, it includes: dividing users into multiple groups according to user attributes, calculating the mean of the user pre-trained representation vector of each group respectively, as the deviation vector of the corresponding group, calculating the weighted sum of the deviation vectors of all groups, obtaining the overall deviation vector of the user attributes, orthogonalizing the overall deviation vector to obtain an orthogonalized deviation vector, and subtracting the user pre-trained representation vector from the orthogonalized deviation vector to obtain a debiased user representation vector. Build a recommendation model, take the debiased user representation vector and the pre-trained item representation vector as input, and train the recommendation model. According to the trained recommendation model, predict the user-item rating. According to the rating prediction results, filter out a candidate recommendation list. In this application, a neighborhood aggregation mechanism is used to learn the representations of users and items, making full use of user-item interaction information. Debiasing processing is introduced to eliminate the bias caused by user attributes by constructing a bias vector, and learn a fairer user representation. Inputting the debiased user representation and item representation into the recommendation model can improve the fairness and accuracy of the recommendation.

[0010] Furthermore, we construct the adjacency matrix A of user-item interaction: we regard users and items as two types of nodes in a bipartite graph. If user u interacts with item i (such as rating, clicking, etc.), the corresponding position A[u, i] in the adjacency matrix A is marked as 1, otherwise it is marked as 0. The adjacency matrix A captures the direct relationship of user-item interaction. We perform exponential weighting on the elements in the adjacency matrix A to obtain the weighted adjacency matrix A': we perform exponential weighting on each non-zero element A[u, i] in the adjacency matrix A, that is, A'[u, i] = eA[u,i] . Exponential weighting can widen the weight difference between interactive and non-interactive edges, highlighting the importance of interactive relationships. The obtained weighted adjacency matrix A' emphasizes the interaction strength between users and projects. According to the weighted adjacency matrix A', node representations are aggregated and updated: Graph Neural Network is used to learn node representations. Aggregation phase: For each node, the representation vectors of its neighboring nodes are aggregated. For example, for user node u, the representation vectors of the project nodes with which it has interacted are aggregated. Update phase: Use the aggregated neighbor information to update the node's own representation vector. Feedforward neural networks and other methods can be used for updating. Through multiple rounds of iterative aggregation and updating, pre-trained representation vectors of user nodes and project nodes are obtained. In this application, the graph structure is used to represent the user-project interaction relationship, which naturally models the mutual influence of users and projects. The neighborhood aggregation mechanism can learn the local structural information of nodes, making similar users or projects closer in the representation space. Exponential weighting highlights the importance of interactive behavior and makes learning more focused on the observed interaction signals. The pre-trained user and project representation vectors contain rich collaborative filtering information and can be used for downstream recommendation tasks.

[0011] Further, the technical solution for constructing the user-item interaction adjacency matrix A is as follows: Construct the user-item interaction matrix R: Define a two-dimensional matrix R, where rows represent users and columns represent items. Traverse the interaction records of each user u and item i: If user u has interacted with item i, then replace the corresponding position R in the matrix R with ui =1; otherwise, R ui =0. The resulting matrix R is the user-item interaction matrix, which represents the interaction relationship between users and items. Based on the interaction matrix R, calculate the adjacency matrix A: Transpose the matrix R to obtain the transposed matrix RT. The transpose operation swaps the rows and columns of the matrix. T Multiply them together to get the adjacency matrix A. The calculation formula is: Rules for matrix multiplication: Elements A in A ij Represents the number of items that user i and user j have interacted with together. Diagonal element A ii Represents the total number of items that user i has interacted with. In this application, an adjacency matrix A of user-item interaction is constructed. This matrix represents the similarity between users, that is, the number of items that users have interacted with together. The adjacency matrix A can be used in subsequent recommendation algorithms, such as user-based collaborative filtering recommendations.

[0012] Furthermore, the purpose of exponentially weighting the elements in the adjacency matrix A is to enhance the influence of the number of common interaction items between users and highlight the similarity between users. It includes: performing an exponential operation on each element in the adjacency matrix A: defining an exponential weighting parameter α to control the strength of the exponential operation. Traversing each element Aij in the adjacency matrix A: performing an exponential operation on the element Aij: ij Perform exponential operation to obtain the weighted value Preferably, the exponentially weighted adjacency matrix A is normalized to scale the elements of each row or column to the range of [0, 1]. Different methods can be used for normalization, such as maximum and minimum value normalization, L1 norm normalization, etc. Normalization can balance the weight differences between different users so that they can be compared on the same scale. By exponentially weighting the elements in the adjacency matrix A, we can control the degree of influence of the number of common interaction items between users on similarity. Exponential weighting can amplify or reduce the original similarity value, thereby giving different importance in the recommendation algorithm.

[0013] Furthermore, a technical solution for performing an exponential operation on each element in the adjacency matrix A and iteratively updating the representation vector matrix of users and items includes: initializing the representation vector matrix E(0) of users and items: defining a matrix E(0), wherein rows represent users and items, and columns represent the dimensions of the vectors. The element values ​​of the matrix E(0) can be randomly initialized, or other initialization methods such as Xavier initialization or He initialization can be used. The initialized matrix E(0) represents the initial position of users and items in the representation vector space. Calculating the degree matrix D: The degree matrix D is a diagonal matrix, and the diagonal elements are the degrees of each node. The degree of a node represents the number of edges connecting the node to other nodes. For the adjacency matrix A of user-item interactions, the degree can be obtained by summing the elements of each row or column. Place the degree values ​​on the diagonal of the degree matrix D, and set other positions to 0. Iteratively update the representation vector matrix E(k+1) of users and items: in, is the exponentially weighted adjacency matrix, and β is the exponential weight parameter, ranging from 0 to 1. A represents the square root of the inverse matrix of the degree matrix D, which can be obtained by performing the square root operation on the diagonal elements of the degree matrix D. βIt represents exponential operation on each element of the representation vector matrix A, that is, raising each element to the power of β. The result of matrix multiplication is assigned to E(k+1) to obtain the updated representation vector matrix. Repeat the iterative update process until the preset number of iterations is reached or the convergence condition is met. The representation vector matrix E(k) after iterative convergence is used as the pre-trained representation vector of users and projects. The pre-trained representation vector can be used for subsequent tasks, such as recommendation systems, link prediction, etc. In the present application, the pre-trained representation vectors of users and projects are obtained by iterative updating using the adjacency matrix A and the degree matrix D. This method combines the structural information of the graph and the interactive relationship between nodes, and makes similar users and projects closer in the representation vector space by iteratively propagating and updating the representation vector.

[0014] Further, after performing an exponential operation on each element in the adjacency matrix A, a threshold process is performed to obtain a weighted adjacency matrix A', including: performing an exponential operation on each element in the adjacency matrix A: defining an exponential weighting parameter α to control the intensity of the exponential operation. Traversing each element A in the adjacency matrix A ij : For element A ij Perform exponential operation to obtain the weighted value Exponential operation can be achieved by the following formula: Effect of exponential operation: When α>1, the exponential operation will increase the original value, and the weights between users with higher similarity will be further enhanced. When 0<α<1, the exponential operation will reduce the original value, and the weights between users with lower similarity will be further weakened. When α=1, the exponential operation will not change the original value, which is equivalent to not performing exponential weighting. Threshold processing is performed on the adjacency matrix A after the exponential operation to obtain the weighted adjacency matrix A": a threshold parameter δ is defined to control the minimum value of the elements in the adjacency matrix. Traverse each element A in the adjacency matrix A after the exponential operation ij :The element A ij Compare with threshold δ: If A ij is greater than or equal to δ, then A ij The value of remains unchanged. If A ij If Aij is less than δ, the value of Aij is set to δ. Threshold processing can be achieved by the following formula: ij '=max(A ij ,δ), the threshold δ defines the minimum value of the elements in the adjacency matrix, which can filter out some connections with low similarity. By setting a suitable threshold δ, the sparsity and connectivity of the graph can be controlled, and some unimportant edges can be removed. After exponential operation and threshold processing, the weighted adjacency matrix A' is obtained. The expression of the weighted adjacency matrix A' is as follows: The weighted adjacency matrix A' combines the effects of exponential weighting and threshold processing, enhancing the weights of important connections while filtering out some unimportant connections. Each element in the adjacency matrix A is subjected to exponential operation and threshold processing to obtain the weighted adjacency matrix A'. This process can be seen as optimizing and adjusting the graph structure, highlighting important connections while removing some noise connections.

[0015] Furthermore, the user pre-trained representation vector is debiased, including: dividing the user into multiple groups according to the user attributes: determining the user attributes used to divide the groups, such as gender, age, region, etc. Define corresponding groups for each user attribute, for example, gender can be divided into two groups of male and female. According to the user's attribute information, each user is assigned to the corresponding group. The mean of the pre-trained representation vector of each group of users is calculated respectively as the deviation vector of the corresponding group: for each group, the pre-trained representation vectors of all users in the group are collected. The mean of the pre-trained representation vectors of the users in the group is calculated, that is, the average value is calculated for each dimension. The calculated mean vector is used as the deviation vector of the group, indicating the average representation deviation of the users in the group. The weighted sum of the deviation vectors of all groups is calculated to obtain the overall deviation vector of the user attributes: for the deviation vector of each group, a weight coefficient is assigned to indicate the importance of the group in the overall deviation. The deviation vectors of all groups are multiplied by their corresponding weight coefficients, and the results are added. The obtained vector is the overall deviation vector of the user attributes, indicating the comprehensive effect of the deviations of different groups. The obtained overall deviation vector is orthogonalized to obtain an orthogonalized deviation vector: the overall deviation vector is orthogonalized to make it orthogonal to the user pre-trained representation vector space. Orthogonalization can be achieved through the Gram-Schmidt orthogonalization process: the overall deviation vector is used as the initial vector for orthogonalization. The basis vectors of the user pre-trained representation vector space are iterated: the projection of the current basis vector and the initial vector is calculated. The projection result is subtracted from the initial vector. The updated vector is normalized. After iterative processing of all basis vectors, an orthogonalized deviation vector is obtained. The user pre-trained representation vector is subtracted from the orthogonalized deviation vector to obtain a debiased user representation vector: for each user's pre-trained representation vector, it is subtracted from the orthogonalized deviation vector. The result obtained is the debiased user representation vector, which removes the overall deviation effect of the user attributes. Technical solution for debiasing the user pre-trained representation vector. This process takes into account the influence of user attributes, and finally obtains the debiased user representation vector by calculating the deviation vectors of different groups and orthogonalizing them. Debiasing can reduce the impact of user attributes on the representation vector, making the representation vector more fair and unbiased.

[0016] Furthermore, the mean of the pre-trained representation vectors of users in each group is calculated separately, including: calculating the mean of the pre-trained representation vectors of users for each group: dividing users into multiple groups according to user attributes, such as male group, female group, etc. For each group, collect the pre-trained representation vectors of all users in the group. Calculate the mean of the pre-trained representation vectors of users in the group, that is, find the average value for each dimension. The calculated mean vector is used as the deviation vector of the group, which represents the average representation deviation of users in the group. Calculation of the mean of the pre-trained representation vectors of male group users: Assume that there are N male group users. m The dimension of the user pre-trained representation vector is d. Define the deviation vector v for the male group m , whose dimension is also d. For each user i in the male group, obtain its pre-trained representation vector u i Sum all pre-trained representation vectors of users in the male group and divide the sum by the number of users in the male group N m , and obtain the deviation vector v of the male group m , the calculation formula is as follows: v m Each dimension of represents the average value of users in the male group on that dimension. Mean calculation of pre-trained representation vectors of users in other groups: For other groups, such as the female group, the calculation process is the same as that of the male group. Assume that there are N users in the female group f The dimension of the user pre-trained representation vector is d. Define the deviation vector v for the female group f , whose dimension is also d. Sum all the pre-trained representation vectors of users in the female group and divide by the number of users in the female group N f , and obtain the deviation vector v of the female group f . Get the deviation vector of each group: After the above calculation, we get the deviation vector of each group, such as the deviation vector v of the male group m , the deviation vector v for the female group f Etc. These deviation vectors represent the average deviation of different groups of users in the pre-trained representation vector space. This application takes into account the characteristics of different user attribute groups, and by calculating the average deviation of each group, the overall deviation of different groups of users in the pre-trained representation vector space can be captured.

[0017] Further, the overall deviation vector of the user attributes is calculated, including: dividing the users into a male group, a female group and an uncertain group according to the user attributes. For the male group, the mean vm of the pre-trained representation vectors of the users in the group is calculated. For the female group, the mean v of the pre-trained representation vectors of the users in the group is calculated. f For the uncertain group, calculate the mean v of the pre-trained representation vectors of the users in the group un. According to the deviation vector v of the male group and the female group m and v f , calculate the first deviation vector v bias1 :v bias_1 =v f -v un ; Deviation vector v according to male group and uncertain group m and v un , calculate the second deviation vector v bias2 :v bias_2 =v un -v m ; Deviation vector v according to female group and uncertain group f and v un , calculate the third deviation vector v bias3 :v bias_3 =vm-v f ; The three deviation vectors v bias1 、v bias2 and v bias3 Perform weighted summation to obtain the overall deviation vector v of user attributes bias :v bias =λ1×v bias1 +λ2×v bias2 +λ3×v bias3 , where λ1, λ2, and λ3 are weight coefficients of the three deviation vectors, indicating the importance of different deviation vectors in the overall deviation. The value of the weight coefficient can be set according to actual needs and experience, satisfying λ1+λ2+λ3=1. After the above calculation, we get the overall deviation vector v of the user attribute bias The overall deviation vector v bias The deviation information between different groups is integrated to show the overall deviation effect of user attributes on the pre-trained representation vector. The overall deviation vector v bias of user attributes is calculated. Among them, the three deviation vectors v bias1 、v bias2 and v bias3 The deviations between the male group and the female group, the male group and the uncertain group, and the female group and the uncertain group are respectively represented. This application obtains the overall deviation vector of user attributes by calculating the deviations between different groups and performing weighted summation. The overall deviation vector reflects the overall impact of different user attributes on the pre-trained representation vector, providing key information for subsequent debiasing processing.

[0018] Further, the user pre-trained representation vector is subtracted from the orthogonalized bias vector to obtain a debiased user representation vector, including: taking the overall bias vector v bias of the user attributes as the initial vector of orthogonalization, performing Gram-Schmidt orthogonalization on the overall bias vector of the user attributes to obtain an orthogonalized bias vector; and calculating the debiased user representation vector according to the following formula: , where α is an adjustable parameter and v bias_j is the orthogonalized deviation vector, <e[i],v bias_j > is the inner product operation of the pre-trained user vector and the bias vector, and j represents the jth orthogonalized bias vector. The user pre-trained representation vector is subtracted from the orthogonalized bias vector to obtain the debiased user representation vector. The Gram-Schmidt orthogonalization process ensures that the bias vector is orthogonal to the user representation vector space and eliminates the projection of the bias vector in the user representation vector space. By subtracting the user representation vector from the orthogonalized bias vector, the overall bias effect of the user attributes is effectively removed, and a more fair and unbiased user representation vector is obtained. The debiased user representation vector can be used in subsequent recommendation tasks to improve the fairness and accuracy of the recommendation.

[0019] 3. Beneficial effects

[0020] Compared with the prior art, the advantages of this application are:

[0021] The neighborhood relationship graph is constructed using user-item interaction information, and the node representation is aggregated and updated iteratively through the weighted adjacency matrix and degree matrix to obtain the initial user and item representation vectors, which fully integrates the neighborhood topological structure information.

[0022] An exponential weighting mechanism is introduced in the neighborhood aggregation process to suppress high-degree nodes and alleviate the unfairness of recommendations caused by long-tail items. Project fairness is considered in the pre-training stage, which is conducive to subsequent fair recommendations.

[0023] The users are divided into groups according to their attributes, the deviation vector of each group is calculated, the weighted sum of the deviation vectors of all groups is taken to get the overall deviation vector, and the overall deviation vector is orthogonalized to eliminate potential user attribute bias, thus effectively alleviating the problem of user group bias in the recommendation process.

[0024] The node degree adjustment strategy in the pre-training stage improves project fairness, and the user debiasing processing in the fine-tuning stage enhances user fairness. The organic combination of innovative measures in the two stages achieves a trade-off between fairness and accuracy.

[0025] A large number of comparative experiments were carried out on two real-world datasets. Compared with the existing fair recommendation model, this method significantly improved the fairness of the recommendation system while maintaining high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of a fair recommendation method according to an embodiment of the present application;

[0027] Figure 2 is a performance comparison chart of one embodiment of the present application and other models;

[0028] Figure 3 is a block diagram of a fair recommendation system according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The present application is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0030] Example 1

[0031] Figure 1 is a flowchart of a fair recommendation method according to an embodiment of the present application. Figure 1 As shown, the method includes the following steps: a recommendation method based on fair representation learning, including: obtaining a set of items with historical user interactions and a set of users with historical interactions on each item as user-item neighborhood information; pre-training the representation vectors of the user set and the item set using a neighborhood aggregation mechanism based on the obtained neighborhood information to obtain a user pre-trained representation vector and an item pre-trained representation vector; de-biasing the user pre-trained representation vector to obtain a de-biased user representation vector; constructing a recommendation model, taking the de-biased user representation vector and the item pre-trained representation vector as input, and training the recommendation model; predicting user-item ratings based on the trained recommendation model; and screening a candidate recommendation list based on the rating prediction results.

[0032] Obtain the neighborhood information of users and items, and perform pre-training based on the neighborhood aggregation mechanism: Construct the user-item interaction matrix R, which is M×N dimensional, with M and N being the number of users and items respectively. If user u has interacted with item i, then Rui=1, otherwise Rui=0. Obtain the user-item adjacency matrix A based on the interaction matrix R. The adjacency matrix A sets the corresponding positions of non-zero elements in the matrix R to 1, and the remaining positions to 0. Based on the adjacency matrix A, the neighborhood aggregation mechanism is used to pre-train the representation vectors of users and items. The graph convolution network (GCN) model is used to improve the richness of node representation by capturing more levels of neighbor information. The specific method is to repeatedly perform graph convolution operations to aggregate the features of neighbor nodes into the new representation of the target node. The specific steps of the pre-training process are: Initialize the representation vector matrix E(0) of users and items, and iteratively update E(k+1) based on the adjacency matrix A and the degree matrix D. E(k+1)=σ(A'E(k)W(k)), where A' is the weighted adjacency matrix, D is the degree matrix, and W(k) is the weight of the kth layer. The iteratively converged E(k) is used as the pre-trained representation vector output for users and items. The user-item neighborhood information is obtained, and the user and item representation vectors are effectively pre-trained based on the neighborhood aggregation mechanism (such as GCN), laying the foundation for the subsequent fair recommendation process.

[0033] During the pre-training process, the node degree is adjusted to alleviate the long-tail distribution problem of items in the recommendation system. The elements in the adjacency matrix A are exponentially weighted, and each element aij in the adjacency matrix A is subjected to an exponential operation: aij'=aij^β, where β is the exponential weight parameter with a value range of 0<β<1. When 0<β<1, the exponential operation will strengthen the smaller elements aij and reduce the larger elements aij. This adjustment helps to balance the size differences of the elements in the adjacency matrix A. After exponential weighting, a fixed threshold is added to each element in the adjacency matrix A', aij'=aij'+ε (ε is a fixed threshold), and a fixed value ε is added to each element of the adjacency matrix A to make β effective. This step further enhances the small elements and weakens the influence of large elements. Based on the weighted adjacency matrix A' and the degree matrix D, the node representation is updated, E(k+1)=σ((A'+I)E(k)W(k)), where A' is the weighted adjacency matrix, I is the identity matrix, W(k) is the kth layer weight, and σ is a nonlinear activation function, such as ReLU, to obtain the final embedding matrix E(k+1) used for prediction. The user part and the project part correspond to the user representation and the project representation, respectively. The embedding matrix is ​​input into the multi-layer perceptron for fine-tuning to adapt to the recommendation task. By exponentially weighting the adjacency matrix in the pre-training stage, the importance of low-degree nodes (usually corresponding to long-tail projects) is strengthened, thereby alleviating the long-tail distribution problem of projects in the recommendation system, improving the exposure rate of long-tail projects, and enhancing the diversity of recommendations.

[0034] Fine-tune the pre-training results, debias the user's attributes, and divide the users into multiple groups according to their attributes. Taking gender as an example, they can be divided into male user group, female user group and uncertain user group. Calculate the mean vector of the pre-trained representation vector of each group of users. For the i-th group, the mean vector calculation formula is: where is the pre-trained representation vector of the j-th user in the i-th group, is the number of users in the i-th group, and the mean vector of the male user group, the mean vector of the female user group and the mean vector of the uncertain group are obtained. Calculate the overall deviation vector. The overall deviation vector calculation formula is: where is the mean vector of the i-th group, is the proportion of the number of users in the i-th group to the total number of users, orthogonalize the overall deviation vector, and use the Gram-Schmidt orthogonalization method to obtain the orthogonalized deviation vector. Debias the user representation vector. The debiased user representation vector calculation formula is: where is the user pre-trained representation vector, is the orthogonalized deviation vector, and is an adjustable parameter. It can effectively remove attribute bias in user representation and eliminate potential bias or discrimination against certain user groups in the algorithm. Orthogonalization further ensures the generalizability of the debiased user representation vector. Fine-tuning based on this debiased representation can maximize the assurance that the model's predictions and recommendations for all user groups are fair and unbiased.

[0035] According to the fine-tuned user-item representation prediction score, the candidate recommendation list is screened. In the pre-training stage, the cross entropy loss is used as the pre-training target of the recommendation task, and the inner product function is used to calculate the user's predicted score for the item: Where u is the user representation vector and v is the item representation vector. Cross entropy loss function: Where R is the training set, (u, i) is the user-item pair, and yui is the actual score (0 or 1). In the fine-tuning stage, the main loss function of the recommendation system (cross entropy loss) is retained. A differentiated fairness regularization term based on Dirichlet distribution is added: where n is the number of items, yui is the predicted score, and yu is the average score of the group to which the u-th user belongs. Overall loss function: L = cross entropy loss + λ*fairness regularization term, λ is an adjustable hyperparameter that controls the strength of the fairness constraint. Prediction and sorting, for the target user u, based on its representation vector u, calculate the predicted scores of all items i: According to the prediction score Sort from high to low, and take the top K as the candidate recommendation list. Using the user-item representation obtained by pre-training and combining it with the fairness constraints in the fine-tuning stage, a recommendation list that is personalized and fair can be generated. The cross entropy loss ensures the relevance of the recommendation, while the fairness regularization term makes the recommendation results for different user groups more fair. Finally, the recommendation list is filtered based on the predicted score, taking into account both accuracy and fairness.

[0036] Experimental evaluation was carried out on two public datasets, ML-100k and ML-1m, as shown in Table 1 below.

[0037] Table 1

[0038] Dataset Number of users Number of projects Number of interactions ml-100k 943 1682 100000 ml-1m 6040 3706 1000208

[0039] The two datasets are divided into training / validation / test sets in a ratio of 8:1:1. Evaluation indicators: Common evaluation indicators such as Top-K accuracy and NDCG measure the relevance of recommendations. Fairness indicators include statistical tests and entropy regularization indicators. Comparison methods: NFCF: Neural Fair Collaborative Filtering, which reduces gender bias when recommending sensitive items. FairGo: Adversarial learning maps embeddings to unbiased space. FOCF: Adding fairness regularization terms to collaborative filtering. PFCN: Counterfactual fairness based on adversarial learning. Experimental process: The proposed method and the baseline method are trained on the ml-100k and ml-1m datasets respectively. During the training process, user / item representations are learned based on interaction data. In the test phase, the accuracy and fairness indicators of recommendations of different methods are evaluated on the test set, and the performance of the proposed method and the baseline method on different indicators is compared. By evaluating on two public datasets of different sizes, the effectiveness and superiority of the fair recommendation method proposed in this paper can be more comprehensively tested, proving that it can achieve a good balance between recommendation accuracy and fairness.

[0040] Explanation of evaluation indicators: hit@5: hit rate, which measures whether the Top-5 recommendation list contains items that users actually like. The larger the value, the better the recommendation relevance. Gini index: describes the degree of unfairness in the distribution of items in the recommendation list. The smaller the value, the more even and fair the distribution. ValueUnfairness: describes the degree of unfairness based on sensitive attributes (such as gender). The smaller the value, the fairer it is. AbsoluteUnfairness: measures the difference in prediction errors between different user groups. The smaller the value, the fairer the model is to different groups.

[0041] Table 2 lists the main results of the experiment, including the comparison of recommendation performance and fairness of all baseline models.

[0042] Table 2

[0043]

[0044] Model comparison and analysis show that the hit@5 value of our model on both datasets is the largest, indicating that the recommendation relevance is optimal. The three fairness index values ​​of our model are also the smallest, showing good fairness. Although FairGo has good fairness indicators, its recommendation accuracy is low, and FOCF has slightly poor fairness capabilities, mainly because traditional CF cannot mine high-order relationships like graph models. PFCN and NFCF have achieved a relative balance between recommendation accuracy and fairness. Overall, our model can achieve the best in both recommendation accuracy and multiple fairness indicators, achieving the organic unity of diversity, personalization, and fairness of the recommendation system. Compared with the baseline method, our model significantly improves the overall performance of the recommendation system. The experimental results fully verify and support the innovation and effectiveness of the method proposed in this paper.

[0045] According to the above data, we can conclude that the advantages and disadvantages of the FairGo and PFCN adversarial training models are: the advantage is that they can achieve the fairness of recommendations well, the disadvantage is that the recommendation hit rate is sacrificed, and the performance varies greatly on different data sets, which is affected by the instability of training. Analysis of the NFCF pre-training + fine-tuning model: Compared with the method in this paper, NFCF ignores the user-item interaction information during pre-training, which is not conducive to improving the recommendation performance. NFCF has no compensation measures in the debiasing process, so the recommendation performance is lost. The advantages of the model in this paper: During pre-training, the node degree is adjusted for training instead of adding non-interactive items, thereby retaining the recommendation hit rate. During fine-tuning, the bias vector is orthogonalized to avoid introducing additional correlations between user vectors and affecting the recommendation results. The above two designs make the model perform well in recommendation indicators such as hit@5. The innovation of the model in this paper: the node degree adjustment strategy in the pre-training stage is innovative, and the innovative method of orthogonalizing the bias vector during fine-tuning. The above innovations help balance the accuracy and fairness of recommendations. In general, the model in this paper combines the innovative strategies of pre-training and fine-tuning to achieve high accuracy and good fairness of the recommendation system, showing excellent overall performance. The two-stage innovations constitute the core innovation of this method and are the key to its excellent experimental results.

[0046] The experimental results are shown in Table 3: Multiple fairness indicators of variant 1 are higher than the final model, the Gini coefficient of variant 2 is lower than the model, and variant 3 achieves better user fairness and reduces value unfairness and absolute unfairness. In addition, in the process of achieving fairness, the hit rate indicator decreases slightly, which is consistent with the theoretical analysis.

[0047] Table 3

[0048] Model hit@5↑ Gini↓ Value↓ Absolute↓ Variant 1 0.8171 0.9753 0.1262 0.1065 Variant 2 0.8077 0.9698 0.1285 0.1068 Variant 3 0.8003 0.9708 0.1184 0.0961 Our Model 0.8018 0.9659 0.1179 0.0965

[0049] Figure 2 From the experimental results in Table 3, we can draw the following analysis: the performance changes under different K values ​​( Figure 2 ), the hit@K hit rate of our model remains optimal under different K values. At the same time, the values ​​of the three fairness indicators are also maintained at a low level, which shows that our model can well weigh and balance the accuracy and fairness of recommendations. Analysis of ablation experiment results (Table 3), Variant 1 (pre-training stage): poor fairness indicators, Variant 2 (pre-training + long-tail item processing): Gini index (item fairness) is improved, Variant 3 (pre-training + fine-tuning denoising): value unfairness and absolute unfairness (user fairness) are improved, and the final model of this paper integrates the above two modules to achieve the optimal hit rate and three fairness indicators. Analysis of the module action mechanism, the long-tail item distribution solution helps to improve the fairness of item distribution (the Gini index is reduced), and the user gender orthogonal denoising can enhance user fairness (value / absolute unfairness is reduced). The two modules are reasonably designed and complementary, which promotes the accuracy and fairness of the overall recommendation system. The experimental results are highly consistent with the theoretical analysis, which fully verifies the effectiveness of the proposed method. While pursuing fairness, it also retains the recommendation accuracy to the greatest extent, achieving a balance between the two. Based on the above analysis, it can be seen that the proposed method designs innovative fairness solutions in the pre-training and fine-tuning stages, and achieves all-round fair recommendations through processing at both the user and project ends. A large number of empirical results show that this method can not only significantly improve the fairness of the recommendation system, but also maintain a high level of accuracy while ensuring good fairness, fully demonstrating the superior performance of this method.

[0050] Example 2

[0051] The fair recommendation system of this application has a pre-training module, which establishes a neighborhood relationship graph based on users, projects and their interactive relationships, performs representation learning based on neighborhood relationships, obtains the initial representation vector of the user project, and adjusts the node degree in the user-project bipartite graph during the aggregation of neighborhood information. This node degree adjustment operation can alleviate the recommendation unfairness caused by long-tail projects. The fine-tuning module adjusts and debiases the user vectors obtained from pre-training, eliminates the bias caused by user attributes (such as gender, etc.), ensures fairness to different user groups, and uses methods such as gender orthogonalization denoising to deal with potential user bias and discrimination. The recommendation module, based on the fine-tuned user-project representation vector, calculates the predicted preference score through the inner product of the user-project vector, sorts the candidate projects, and generates a personalized recommendation list for the user. The core innovations of this fair recommendation system include: the node degree adjustment strategy in the pre-training stage to alleviate the unfairness caused by long-tail projects, the user debiasing processing in the fine-tuning stage, such as gender orthogonalization denoising, etc. The overall framework integrates fairness considerations into both the pre-training and fine-tuning stages.

[0052] Example 3

[0053] The present application also proposes a non-temporary computer-readable storage medium for storing a computer program that implements a fair recommendation method. When the program is executed by a processor, the following functions can be specifically implemented: Pre-training module: construct a neighborhood relationship graph based on users, items and their interactive relationships, perform representation learning, obtain the initial embedding vector of the user item, adjust the node degree during the neighborhood information aggregation process, and alleviate the unfairness caused by long-tail items. Fine-tuning module: adjust and debias the user vector obtained by pre-training, eliminate bias caused by user attributes such as gender, ensure fairness to different user groups, and use gender orthogonal denoising and other methods to deal with potential bias and discrimination in user representation. Recommendation module: Based on the fine-tuned user-item embedding vector, the predicted preference score is calculated by the inner product of the user-item vector, and the candidate items are personalized and sorted to generate the final recommendation list.

[0054] The invention of the present application and its implementation methods are described schematically above. The description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention of the present application. The actual structure is not limited to this. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the present invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A recommendation method based on fair representation learning, comprising: Get the set of items that the user has interacted with historically, and the set of users that each item has interacted with historically, as the user-item neighborhood information; According to the obtained neighborhood information, the neighborhood aggregation mechanism is used to pre-train the representation vectors of the user set and the item set to obtain the user pre-trained representation vector and the item pre-trained representation vector, including: Construct the user-item interaction matrix R, where if user u has interacted with item i, then R ui =1, otherwise R ui =0; According to the interaction matrix R, the calculation formula of the adjacency matrix A is as follows: Among them, R T represents the transposed matrix of matrix R; Perform exponential operation on the elements in the adjacency matrix A to achieve exponential weighting, and add a threshold to each element in the exponentially weighted adjacency matrix A to obtain the weighted adjacency matrix A'; Initialize the representation vector matrices E(0) of users and items respectively; According to the weighted adjacency matrix A' and the degree matrix D, iteratively update the representation vector matrix E(k+1) of the user and item respectively; The update formula for the vector matrix E(k+1) is as follows: Where A' is a weighted adjacency matrix, D is a degree matrix, and the degree matrix D is a diagonal matrix. The diagonal elements are the degrees of each node. The nodes represent users and items. The degree of the node is obtained by summing the elements of each row or column of the adjacency matrix A. is the normalized transposed matrix, β is the exponential weight parameter, and the value range of exponent β is 0 to 1; The iteratively converged E(k) is used as the output of the user pre-training representation vector and the item pre-training representation vector; Debiasing the user pre-trained representation vector to obtain a debiased user representation vector, including: Divide users into multiple groups based on user attributes; Calculate the mean of the pre-trained representation vector of each group of users as the deviation vector of the corresponding group. According to the deviation vector of each group, obtain the overall deviation vector of user attributes. Orthogonalizing the obtained overall deviation vector to obtain an orthogonalized deviation vector; Subtract the user pre-trained representation vector from the orthogonalized bias vector to obtain the debiased user representation vector; Build a recommendation model, take the debiased user representation vector and the item pre-trained representation vector as input, and train the recommendation model; According to the trained recommendation model, the user-item rating is predicted; According to the rating prediction results, a candidate recommendation list is obtained by screening.

2. The recommendation method based on fair representation learning according to claim 1, characterized in that: Calculate the mean of the pre-trained representation vectors of each group of users, including: For the male (m) group, the calculation formula for the mean of the user pre-trained representation vector is as follows: Among them, e[i] is the pre-trained user vector, N m represents the number of users in the male group, i represents the i-th user in the male group, v m is the mean of the pre-trained representation vectors for users in the male group.

3. The recommendation method based on fair representation learning according to claim 2, characterized in that: Get the overall deviation vector of user attributes, formula: v bias_1 =v f -v un v bias_2 =v un -v m v bias_3 =v m -v f Among them, v f represents the mean of the pre-trained representation vectors of users in the female group, v un represents the mean of the pre-trained representation vectors of users in the uncertain group, v m is the mean of the pre-trained representation vectors for users in the male group, v bias1 represents the deviation between the male group and the female group; v bias2 represents the deviation between the male group and the uncertain group; v bias3 Represents the deviation between the female group and the uncertain group.

4. The recommendation method based on fair representation learning according to claim 3, characterized in that: Subtract the user pre-trained representation vector from the orthogonalized bias vector to obtain the debiased user representation vector, including: Perform Gram-Schmidt orthogonalization on the overall deviation vector of user attributes to obtain an orthogonalized deviation vector; The debiased user representation vector is calculated according to the following formula: Among them, α is an adjustable parameter, v bias_j is the orthogonalized deviation vector, <e[i],v bias_j > is the inner product operation of the pre-trained user vector and the bias vector, and j represents the jth orthogonalized bias vector.

Images