Collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering

Abstract

The present invention discloses a collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering, which relates to the technical field of collaborative filtering recommendation algorithms and includes the following steps: first, capturing some personalized information, and adding information of multiple dimensions such as project type to the traditional binary relationship between users and projects to form a tensor. The present invention addresses the data sparsity problem in traditional collaborative filtering recommendation algorithms and proposes a collaborative filtering recommendation algorithm based on tensor decomposition and fuzzy clustering. On the one hand, tensor decomposition is used to fill in missing data to reduce its sparsity, and potential information is mined to remove noise; on the other hand, based on the filled matrix, a fuzzy C-means clustering algorithm is used to classify users, reducing the nearest neighbor search space of the target user and improving the scalability of the algorithm. Finally, a traditional recommendation algorithm is used to generate recommendation results in the class where the target user belongs, with excellent recommendation performance.

Description

Technical Field

[0001] The present invention relates to the technical field of collaborative filtering recommendation algorithms, and in particular to a collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering. Background Art

[0002] With the rapid development of internet technology and the advent of the big data era, information overload is becoming increasingly serious, making it increasingly difficult for users to find the information they need within this vast amount of information. Currently, most online movie platforms, such as Xunlei and Douban Movies, utilize recommendation systems to varying degrees to recommend suitable films to users. This demonstrates that personalized recommendation technology can effectively address the issue of information overload. At the same time, researchers are increasingly focusing on the hot topic of how to provide users with efficient and accurate recommendation services.

[0003] Collaborative filtering is a representative algorithm in recommendation systems. It is generally categorized into memory-based and model-based collaborative filtering algorithms. Memory-based collaborative filtering algorithms predict user preferences for items by calculating similarities between users or items. Model-based collaborative filtering algorithms leverage machine learning to predict the rating relationships between blank items and data, and then recommend the highest-rated items to users. Researchers have also integrated multiple recommendation algorithms to propose hybrid recommendation algorithms, thereby improving recommendation accuracy. However, collaborative filtering also faces various challenges, such as cold start, poor scalability, and data sparsity. In the era of big data, with the rapid growth in the number of users and items, on some shopping websites, users rarely rate the items they purchase due to various factors. This results in an extremely sparse user-item rating data matrix. In such situations, the quality of recommendations using traditional collaborative filtering algorithms is significantly reduced.

[0004] In order to solve the problem of data sparsity, a collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering is urgently needed to change this situation. Summary of the Invention

[0005] The purpose of the present invention is to address the shortcomings of the prior art and propose a collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering. The advantage of the collaborative recommendation algorithm is that it has good recommendation performance.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering includes the following steps:

[0008] S1: First, capture some personalized information. Based on the traditional binary relationship between users and projects, add multiple dimensions such as project type to form a tensor.

[0009] S2: Considering users, items, and item categories, a third-order tensor is constructed. Tensor decomposition can fully explore the implicit information of the data. Gradient descent is used to decompose the third-order tensor to obtain user feature matrices, item feature matrices, and item category feature matrices, thereby obtaining missing values ​​and solving the sparsity problem of the tensor.

[0010] S3: Consider the tensor decomposition process as a low-rank approximation problem. If we simply minimize the total error, we can transform the tensor decomposition approximation problem into an unconstrained optimization problem and obtain the optimization formula.

[0011] S4: Based on the above ideas, a sparse tensor filling algorithm based on tensor decomposition is designed to obtain the user feature matrix, item feature matrix, category feature matrix and the filled tensor.

[0012] By adopting the above technical solutions: on the one hand, tensor decomposition is used to fill in missing data, reduce its sparsity, mine potential information, and remove noise; on the other hand, based on the filled matrix, the fuzzy C-means clustering algorithm is used to classify users, reducing the nearest neighbor search space of the target user and improving the scalability of the algorithm. Finally, the traditional recommendation algorithm is used to generate recommendation results in the class where the target user belongs, with excellent recommendation performance.

[0013] The present invention is further configured such that the sparsity decomposition formula of the tensor is: Among them, u mk represents the user factor of the user feature matrix; v nk The item factors representing the item characteristic matrix; c ck A categorical factor representing the item category matrix.

[0014] By adopting the above technical solution: the score estimation value of the tensor R at the position index can be calculated.

[0015] The present invention is further configured such that the optimization formula is

[0016] By adopting the above technical solution: the total error can be minimized by optimizing the formula.

[0017] The present invention is further configured such that the sparse tensor filling algorithm includes a first algorithm and a second algorithm.

[0018] By adopting the above technical solution: the two algorithms can perform calculations according to different conditions, reducing the difficulty of conversion during calculation.

[0019] The present invention is further configured such that the first algorithm takes as input the original tensor R, the number of users m, the number of items n, the iteration step η, and the number of item categories c;

[0020] Output: U, V, C,

[0021] Begin

[0022] 1Initialize the U∈R m×k ,V∈R n×k ,C∈R c×k

[0023] 2For R mnc ≠0do

[0024] 3

[0025] 4

[0026] 5

[0027] 6

[0028] 7 End

[0029] 8 ReturnU、V、C、

[0030] End.

[0031] By adopting the above technical solution: the first line initializes the user feature matrix, item feature matrix and category feature matrix; lines 2 to 7 perform tensor decomposition and fill the sparse tensor through an iterative method; line 8 returns the obtained user feature matrix, item feature matrix, category feature matrix and the filled tensor.

[0032] The present invention is further configured such that the second algorithm inputs a user-item rating dataset A, an item category dataset B, a fuzzy coefficient m1, a clustering number c1, a convergence precision epsm, and a number of iterations t.

[0033] Output: The predicted rating P of the target user u for the unrated item i ui .

[0034] Begin

[0035] 1 m1←2,c1←3,epsm←1.0e-6,t←100,

[0036] 2 R←Data(A,B)

[0037] 3

[0038] 4[S,V',obj]←FCM(U,c1,t,m1,epsm)

[0039] 5 User Category ←max(S)

[0040] 6 C u ←(User Category ,u)

[0041] 7 For v∈C u ∩v≠u∩r vi ≠0

[0042] 8 N←N∪v

[0043] 9 End

[0044] 10

[0045] 11.Return P ui

[0046] End.

[0047] By adopting the above technical solution: Line 1 performs initialization; Line 2 constructs a user-item-category three-dimensional tensor based on the rating matrix and the item category matrix; Line 3 fills the sparse tensor based on tensor decomposition to obtain the user feature matrix, item feature matrix and category feature matrix; Line 4 performs fuzzy clustering on users based on the obtained user feature matrix; Line 5 obtains the user category matrix by calculating the maximum membership degree; Line 6 finds the category to which user u belongs; Lines 7 to 9 obtain the set of users who have rated item i in the category to which user u belongs; Lines 10 to 11 calculate and return the predicted score of user u for item i.

[0048] The present invention is further configured to calculate u according to the stochastic gradient descent method. mk 、v nk and c ck Perform iterative updates, and the formula is as follows: Where a (a > 0) represents the step size of the gradient descent, also known as the learning rate; S represents the set of observable position indices in the tensor. n,c:(m,n,c)∈S, m,c:(m,n,c)∈S, and m,n:(m,n,c)∈S represent the sets of position indices of all nonzero elements in R(m,:,:), R(:,n,:), and R(:,:,c), respectively.

[0049] By adopting the above technical solution: according to the stochastic gradient descent method, u mk 、vnk and c ck Performing iterative updates can further facilitate minimizing the total error and improve the convenience of calculation.

[0050] The present invention is further configured such that the results calculated by the first algorithm and the second algorithm are both tested and evaluated, and the evaluation calculation formula is:

[0051] Among them, N is the test set size, p ui is the prediction score of the recommendation algorithm, r ui Provide users with real ratings;

[0052] In addition, the accuracy of the recommendation results can also be measured by precision and recall. The larger their values, the better the recommendation effect.

[0053] The accuracy formula is as follows:

[0054]

[0055] The recall formula is as follows:

[0056]

[0057] Among them, P(u) represents the set of items recommended by the system for user u, and T(u) represents the set of items preferred by user u.

[0058] By adopting the above technical solutions, the advantages of the collaborative recommendation algorithm over the existing algorithms can be more intuitively reflected, and at the same time, the rationality of the collaborative recommendation algorithm can be verified.

[0059] The beneficial effects of the present invention are:

[0060] Aiming at the data sparsity problem in traditional collaborative filtering recommendation algorithms, the present invention proposes a collaborative filtering recommendation algorithm based on tensor decomposition and fuzzy clustering. On the one hand, tensor decomposition is used to fill in missing data to reduce its sparsity, and potential information is mined to remove noise. On the other hand, based on the filled matrix, a fuzzy C-means clustering algorithm is used to classify users, reducing the nearest neighbor search space of the target user and improving the scalability of the algorithm. Finally, the traditional recommendation algorithm is used to generate recommendation results in the class where the target user belongs, which has excellent recommendation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a schematic diagram of the process structure of the collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering proposed in the present invention;

[0062] Figure 2This is a schematic diagram of the tensor modeling structure of the collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering proposed in the present invention;

[0063] Figure 3 This is a schematic diagram of the tensor decomposition and filling structure of the collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering proposed in the present invention;

[0064] Figure 4 Schematic diagram of the FCM-based collaborative recommendation structure of the collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering proposed in the present invention.

[0065] Figure 5 This is a structural diagram of the impact of the k value of the collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering proposed in the present invention on the actual prediction results.

[0066] Figure 6 This is a structural diagram of the influence of the number of clusters on the recommendation accuracy of the collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering proposed in the present invention.

[0067] Figure 7 This is a structural diagram of the impact of the number of target user neighbors on the MAE value of the collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering proposed in the present invention.

[0068] Figure 8 This is a structural diagram of the accuracy and recall rate of the collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering proposed in the present invention. DETAILED DESCRIPTION

[0069] The technical solution of this patent is further described in detail below in conjunction with specific implementation methods.

[0070] Reference Figure 1-4 ,The collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering includes the following steps:

[0071] S1: First, capture some personalized information. Based on the traditional binary relationship between users and projects, add multiple dimensions such as project type to form a tensor.

[0072] S2: Considering users, items, and item categories, a third-order tensor is constructed. Tensor decomposition can fully explore the implicit information of the data. Gradient descent is used to decompose the third-order tensor to obtain user feature matrices, item feature matrices, and item category feature matrices, thereby obtaining missing values ​​and solving the sparsity problem of the tensor.

[0073] S3: Consider the tensor decomposition process as a low-rank approximation problem. If we simply minimize the total error, we can transform the tensor decomposition approximation problem into an unconstrained optimization problem and obtain the optimization formula.

[0074] S4: Based on the above ideas, a sparse tensor filling algorithm based on tensor decomposition is designed to obtain the user feature matrix, item feature matrix, category feature matrix and the filled tensor.

[0075] On the one hand, tensor decomposition is used to fill in missing data, reduce its sparsity, mine potential information and remove noise; on the other hand, based on the filled matrix, the fuzzy C-means clustering algorithm is used to classify users, reducing the nearest neighbor search space of the target user and improving the scalability of the algorithm. Finally, the traditional recommendation algorithm is used to generate recommendation results in the class where the target user is located, with excellent recommendation performance.

[0076] In this embodiment, the tensor sparsity decomposition formula is: Among them, u mk represents the user factor of the user feature matrix; v nk The item factors representing the item characteristic matrix; c ck Represents the category factor of the item category matrix. The score estimate of the tensor R at the position index can be calculated using the tensor sparsity decomposition formula.

[0077] In this embodiment, the optimization formula is The total error can be minimized by optimizing the formula.

[0078] In this embodiment, the sparse tensor filling algorithm includes a first algorithm and a second algorithm. The two algorithms can be used to perform calculations for different conditions, thereby reducing the difficulty of conversion during calculation.

[0079] In this embodiment, the first algorithm takes as input the original tensor R, the number of users m, the number of items n, the iteration step η, and the number of item categories c;

[0080] Output: U, V, C,

[0081] Begin

[0082] 1 Initialize the U∈R m×k ,V∈R n×k ,C∈R c×k

[0083] 2 For R mnc ≠0do

[0084] 3

[0085] 4

[0086] 5

[0087] 6

[0088] 7 End

[0089] 8 ReturnU、V、C、

[0090] End.

[0091] Line 1 initializes the user feature matrix, item feature matrix, and category feature matrix; lines 2 to 7 perform tensor decomposition and fill the sparse tensor through iteration; line 8 returns the obtained user feature matrix, item feature matrix, category feature matrix, and the filled tensor.

[0092] In this embodiment, the second algorithm inputs a user-item rating dataset A, an item category dataset B, a fuzzy coefficient m1, a cluster number c1, a convergence precision epsm, and an iteration number t.

[0093] Output: The predicted rating P of the target user u for the unrated item i ui .

[0094] Begin

[0095] 1 m1←2,c1←3,epsm←1.0e-6,t←100,

[0096] 2 R←Data(A,B)

[0097] 3

[0098] 4[S,V',obj]←FCM(U,c1,t,m1,epsm)

[0099] 5 User Category ←max(S)

[0100] 6 C u ←(User Category ,u)

[0101] 7 For v∈C u ∩v≠u∩r vi ≠0

[0102] 8 N←N∪v

[0103] 9 End

[0104] 10

[0105] 11.Return P ui

[0106] End.

[0107] The first line performs initialization; the second line constructs a user-item-category three-dimensional tensor based on the rating matrix and the item category matrix; the third line fills the sparse tensor based on tensor decomposition to obtain the user feature matrix, item feature matrix, and category feature matrix; the fourth line performs fuzzy clustering on users based on the obtained user feature matrix; the fifth line obtains the user category matrix by finding the maximum membership degree; the sixth line finds the category to which user u belongs; the seventh to ninth lines obtain the set of users in the category to which user u belongs who have rated item i; the tenth to eleventh lines calculate and return the predicted rating of user u for item i.

[0108] In this embodiment, u is calculated according to the stochastic gradient descent method. mk 、v nk and c ck Perform iterative updates, and the formula is as follows: Where a (a>0) represents the step size of gradient descent, also known as the learning rate; S represents the set of position indices that can be observed in the tensor. n,c:(m,n,c)∈S, m,c:(m,n,c)∈S, m,n:(m,n,c)∈S represent the set of position indices of all non-zero elements on R(m,:,:), R(:,n,:), R(:,:,c) respectively. By using the stochastic gradient descent method to calculate u mk 、v nk and c ck Performing iterative updates can further facilitate minimizing the total error and improve the convenience of calculation.

[0109] In this embodiment, the results calculated by the first algorithm and the second algorithm need to be tested and evaluated. The evaluation calculation formula is:

[0110] Among them, N is the test set size, p ui is the prediction score of the recommendation algorithm, r ui Provide real ratings for users.

[0111] In addition, the accuracy of recommendation results can also be measured by precision and recall. The larger their values, the better the recommendation effect.

[0112] The accuracy formula is as follows:

[0113]

[0114] The recall formula is as follows:

[0115]

[0116] Here, P(u) represents the set of items recommended by the system for user u, and T(u) represents the set of items preferred by user u. Experimentation and evaluation can more intuitively demonstrate the advantages of collaborative recommendation algorithms over existing algorithms and verify their rationality.

[0117] In the first experiment, collaborative filtering recommendation is performed based on the data filled with tensor decomposition. The influence of different k values ​​(tensor rank) on the actual prediction results is tested. The value range of k is set to [10, 60]. The results are evaluated by MAE. The experimental results refer to Figure 5 shown.

[0118] As the figure shows, when decomposing the tensor at different k values, the MAE decreases as the k value increases. This means that the recommendation accuracy improves with increasing k, reaching its highest accuracy when k = 50. Therefore, choosing 50 as the tensor rank reduces the algorithm's computational complexity while ensuring recommendation accuracy.

[0119] The recommendation effect of the algorithm in this paper is not only related to the rank of the tensor, but also depends on the size of the number of clusters c1. Therefore, in the second experiment, it is necessary to study the impact of the number of clusters on the recommendation accuracy. At the same time, explore the advantages of FCM clustering compared with traditional K-Means clustering and hierarchical clustering. First, let k = 50, c1 = 3, and increase the number of clusters by 1 each time until c1 = 8, and obtain the corresponding MAE values ​​in turn to explore the impact of the number of clusters on the recommendation accuracy, such as Figure 6 shown.

[0120] As can be seen from the figure, among the three clustering algorithms, the FCM algorithm has the best clustering effect overall, and when c1=4, it achieves its global optimal point, the MAE value is the smallest, and the recommendation accuracy is the highest.

[0121] In order to verify the performance of the algorithm in this paper, the algorithm CF-TDFC proposed in this paper is compared with the following four algorithms for experimental analysis.

[0122] (1) User-CF

[25] : a recommendation algorithm based on user collaborative filtering.

[0123] (2) Item-CF

[26] : Item-based collaborative filtering recommendation algorithm.

[0124] (3) Slope-CF

[19] : Research on collaborative filtering algorithm based on slope-one algorithm to improve rating matrix filling.

[0125] (4) SK-HCF

[27] : A hybrid recommendation algorithm based on SVD filling and user attribute feature clustering.

[0126] The above five algorithms were compared on the Movielens dataset. The neighbor interval was set to [10,70] and the interval was 10. The MAE value was used to measure the recommendation accuracy to explore the impact of the number of neighbors on MAE and obtain the optimal solution. The smaller the value, the higher the recommendation accuracy.

[0127] Depend on Figure 7 Experimental data show that, overall, the MAE values ​​of the four collaborative filtering recommendation algorithms (User-CF, Slope-CF, SK-HCF, and CF-TDFC) gradually decrease as the number of neighbors of the target user increases, and they achieve global optimal solutions when the number of neighbors is 60, 60, 40, and 50, respectively. The MAE value of Item-CF is independent of the number of neighbors. Furthermore, regardless of the number of neighbors, the proposed collaborative filtering recommendation algorithm based on tensor decomposition and fuzzy C-means clustering achieves the lowest MAE value. This is primarily because the CF-TDFC algorithm utilizes tensor decomposition to fill in missing data, effectively alleviating data sparsity and improving the accuracy of rating predictions. In terms of the MAE metric, it achieves superior accuracy compared to the Item-CF, User-CF, Slope-CF, and SK-HCF algorithms.

[0128] To further illustrate the effectiveness of the experiment, the accuracy of the recommendation results is evaluated using precision and recall based on the Movielens dataset. The larger the precision and recall values, the better the recommendation quality.

[0129] from Figure 8 It can be seen that the accuracy and recall rates of the Item-CF and User-CF algorithms are not much different. The Slope-CF and SK-HCF algorithms have a certain improvement in recommendation efficiency compared with traditional algorithms. The accuracy and recall rates of the CF-TDFC algorithm proposed in this paper are better than the other four algorithms in the figure. This is because the CF-TDFC algorithm not only uses tensor decomposition to alleviate the disadvantages of extremely sparse tensors, but also uses the FCM algorithm to reduce the nearest neighbor search range, effectively alleviating the scalability problem caused by the huge number of users in the similarity calculation process, reducing the computational complexity, and improving the algorithm recommendation quality.

[0130] In order to evaluate the time complexity of the algorithm, the time performance of the algorithm is compared and analyzed based on the model training time of the algorithm in this paper and the comparison algorithm.

[0131]

[0132]

[0133] The above data shows that the CF-TDFC algorithm takes a long time to train its model, as it first requires tensor decomposition and fuzzy clustering, as well as iterative operations on the user, item, and category feature matrices. The SK-HCF algorithm, which primarily involves SVD decomposition and k-means clustering, as well as iterative operations on the user and item feature matrices, takes a similar amount of time to train. Regarding online prediction time, all five algorithms are very similar and take very short times.

[0134] The above series of comparative experiments show that the CF-TDFC algorithm proposed in this paper has better recommendation performance than other existing algorithms while ensuring time efficiency.

[0135] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering, characterized by: The following steps are involved: S1: First, capture some personalized information. On the basis of the traditional binary relationship between users and items, information about the item category dimension is added to form a tensor. S2: Considering users, items, and item categories, a third-order tensor is constructed. The implicit information of the data is fully mined through tensor decomposition. Gradient descent is used to decompose the third-order tensor to obtain the user feature matrix, item feature matrix, and item category feature matrix, thereby obtaining missing values ​​and addressing the sparsity of the tensor. S3: The tensor decomposition process is regarded as a low-rank approximation problem. The total error is simply minimized, and the approximation problem of tensor decomposition is transformed into an unconstrained optimization problem, and the optimization formula is obtained. S4: Design a sparse tensor filling algorithm based on tensor decomposition, and obtain the user feature matrix, project feature matrix, project category feature matrix and the filled tensor. The sparse tensor filling algorithm includes a first algorithm and a second algorithm. The first algorithm takes as input the original tensor R, the number of users m, the number of projects n, and the iteration step size. And the number of project categories c, line 1 initializes the user feature matrix, project feature matrix and category feature matrix; lines 2 to 7 perform tensor decomposition and fill the sparse tensor through iteration; line 8 returns the obtained user feature matrix, project feature matrix, category feature matrix and the filled tensor; Output: user feature matrix, item feature matrix, category feature matrix and padded tensor; The second algorithm is to input the user-item rating dataset A, the item category dataset B, the fuzzy coefficient m1, the number of clusters c1, the convergence precision epsm, and the number of iterations t. The first line performs initialization; the second line constructs the user-item-category three-dimensional tensor based on the rating matrix and the item category matrix; the third line fills the sparse tensor based on tensor decomposition to obtain the user feature matrix, the item feature matrix, and the category feature matrix; the fourth line performs fuzzy clustering on the users based on the obtained user feature matrix; the fifth line obtains the user category matrix by finding the maximum membership degree; the sixth line finds the category to which user u belongs; the seventh to ninth lines obtain the set of users who have rated item i in the category to which user u belongs; the tenth to eleventh lines calculate and return the predicted rating of user u for item i; Output: The predicted rating of the target user u for the unrated item i ; The sparsity decomposition formula of the tensor is: ,in, User factors representing the user feature matrix; Item factors representing the item characteristic matrix; Category factors representing the item category feature matrix; The optimization formula is .

2. The collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering according to claim 1 is characterized in that: According to the stochastic gradient descent method 、 and Perform iterative updates, and the formula is as follows: , , ,in, Represents the step size of gradient descent, also known as the learning rate; S represents the set of position indices that can be observed in the tensor, Respectively 、 、 The set of position indices of all non-zero elements in .

3. The collaborative recommendation algorithm based on tensor decomposition and fuzzy C-means clustering according to claim 1, characterized in that: The results calculated by the first algorithm and the second algorithm need to be tested and evaluated. The evaluation calculation formula is: , where N is the test set size, Score the recommendation algorithm's predictions, Provide real ratings for users.

Images