A collaborative filtering recommendation method and system based on adaptive noise adding privacy protection
By using DBSCAN and box plots to filter out abnormal users, and combining adaptive noise addition technology with the k-means algorithm, the problem of uneven noise addition in collaborative filtering recommendations is solved, thereby improving recommendation accuracy and privacy protection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING UNIV OF TECH
- Filing Date
- 2024-01-09
- Publication Date
- 2026-07-24
AI Technical Summary
Existing collaborative filtering recommendation algorithms suffer from privacy risks such as uneven noise addition leading to reduced data availability, decreased accuracy of recommendation results, and privacy leakage, making it difficult to provide accurate recommendations while protecting privacy.
The DBSCAN algorithm is used to screen abnormal users, and box plots are used to identify truly abnormal users. Noise of different magnitudes is added based on user similarity values. The k-means algorithm is used to select centroids and iteratively update them. An adaptive noise matrix is constructed to cluster users and finally generate recommendation results.
It improves the accuracy of recommendation results, avoids the accumulation of noise during the iteration process, enhances user privacy protection, and reduces the risk of privacy leakage.
Smart Images
Figure CN117972225B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data recommendation technology, and in particular to a collaborative filtering recommendation method and a collaborative filtering recommendation system based on adaptive noise reduction and privacy protection. Background Technology
[0002] The explosive growth of various types of information data often overwhelms users. On the one hand, users struggle to extract information of interest from a sea of unfamiliar data; on the other hand, when users visit a website or search engine with a specific question, the sheer volume of information they encounter makes it difficult to discern which information is most relevant. Personalized recommendation systems can significantly improve service quality and website access efficiency. Collaborative filtering algorithms are a key technology for achieving personalized recommendations. Their core idea is to collect users' historical behavioral information (such as browsing and rating data) and leverage neighbor groups of users or items with similar behaviors and interests to provide personalized recommendations to selected target users. For example, on shopping websites, by clustering shoppers with products of different values, products of interest to similar users can be recommended, increasing purchase rates and boosting platform and business profits. To build recommendation models, collaborative filtering algorithms typically establish relationships between users or items. The effectiveness of the recommendation depends on the quality of these relationships. To group users or items with high similarity together, collaborative filtering employs clustering algorithms. However, the clustering process involves a large amount of users' private data, and if this data is maliciously used by criminals, it may cause unpredictable harm to users, or even endanger their personal safety and property. Therefore, privacy protection has become the most pressing issue to be addressed.
[0003] Current research on privacy protection in collaborative filtering recommendation algorithms mainly includes: design of recommendation algorithms based on differential privacy, trade-offs between privacy and utility, improvement of noise models, and research on specific data types.
[0004] 1. Differential Privacy-Based Recommendation Algorithm Design: Researchers have proposed several recommendation algorithms based on differential privacy to protect user privacy. These algorithms ensure individual user privacy by adding noise or modifying the dataset, and make recommendations as accurately as possible while protecting privacy.
[0005] 2. The Trade-off Between Privacy and Utility: There is a trade-off between differential privacy protection and the utility of recommendation algorithms. Researchers strive to find a balance that protects user privacy while providing useful recommendation results. They achieve this goal by adjusting privacy parameters and optimizing noise models.
[0006] 3. Improved Noise Models: Noise models are crucial for achieving differential privacy. Researchers have proposed several improved noise models, such as Laplace and Gaussian mechanisms, to enhance the utility and privacy protection capabilities of recommendation algorithms.
[0007] 4. Research on Specific Data Types: Different types of data have varying impacts on privacy protection and recommendation algorithms. Therefore, researchers are also conducting research on specific data types, such as social network data and location data, to achieve better privacy protection and recommendation performance.
[0008] However, existing research on privacy protection in collaborative filtering recommendation algorithms has some shortcomings:
[0009] 1. Differential privacy protection similarity scrambling: Adding the same amount of noise to each similarity level reduces data availability to some extent. The ideal solution is to add more noise to users with high similarity and less noise to users with low similarity.
[0010] 2. The trade-off between privacy and utility: Under differential privacy protection, noise needs to be added to the original data or the dataset needs to be modified to protect user privacy. This may lead to a decrease in the accuracy of the recommendation results, affecting the utility of the recommendation system. How to make recommendations as accurately as possible while protecting privacy remains a challenging problem.
[0011] 3. Privacy Leakage Risk: Although differential privacy can provide a certain level of privacy protection, the risk of privacy leakage still exists. Attackers can obtain users' sensitive information by analyzing recommendation results and making multiple queries. Therefore, it is necessary to further strengthen privacy protection measures to reduce the risk of privacy leakage. Summary of the Invention
[0012] To address the aforementioned issues, this invention provides a collaborative filtering recommendation method and system based on adaptive noise enhancement for privacy protection. It preprocesses the user dataset using a combination of the DBSCAN algorithm and box plots to filter out abnormal user data. By adding noise of varying sizes to each user to construct a similarity noise matrix, it avoids the problem of blindly adding noise of the same size causing significant deviations in cluster centroids. The k-means algorithm is used for initial centroid selection and iterative centroid updates. Finally, recommendation results are generated based on the cluster to which the target user belongs. This approach avoids the accumulation of noise during iteration by adding noise in multiple places, thus achieving user privacy protection in the entire collaborative filtering recommendation algorithm and improving the accuracy of the recommendation results.
[0013] To achieve the above objectives, this invention provides a collaborative filtering recommendation method based on adaptive noise reduction and privacy protection, comprising:
[0014] For the user dataset, the DBSCAN algorithm is used to filter out suspected abnormal users, and the box plot is used to identify the real abnormal users, thereby removing the abnormal user data from the user dataset.
[0015] Calculate the similarity between the user datasets, and add noise of different sizes according to the similarity values to construct a user similarity noise matrix;
[0016] Using the k-means algorithm, initial centroids are selected based on user similarity in the user similarity-plus-noise matrix, and iterative updates are performed to achieve user data clustering;
[0017] For the neighbor set formed by other users in the cluster to which the target user belongs, the items that the target user already owns are deleted from the neighbor set to obtain the recommendation list;
[0018] Calculate the rating for each item in the recommendation list, and recommend a preset number of items with the highest ratings to the target user.
[0019] In the above technical solution, preferably, the specific method of adding noise of different magnitudes according to the similarity values between users is as follows:
[0020] Based on the similarity scores between users, users with higher similarity scores are assigned relatively more noise, while users with lower similarity scores are assigned relatively less noise.
[0021] In the above technical solution, preferably, the step of adding noise of different magnitudes according to the similarity values between each user to construct a user similarity noise matrix specifically includes:
[0022] For the similarity matrix Sim formed by the similarity between users, a privacy budget ε is preset, and user x is calculated. i With user x j The maximum privacy budget between them is [ε / (n-1)]. 2 ];
[0023] Search for user x in the similarity matrix Sim i The minimum similarity between a user and another user is denoted as minSimilarity.
[0024] Calculate user x i With user x j The similarity noise between them is Noise' = Lap(Δf / ε'), where ε' = [ε / (n-1)]. 2 ]*[minSimilarity / Sim(d i ,d j )];
[0025] For user x i With user x j The similarity between Sim(d) i ,d j Adding noise, we get SimM(d) i ,d j ) = Sim(d i ,d j )+Noise';
[0026] Noise is added to the similarity between users to obtain the similarity-noise matrix SimM.
[0027] In the above technical solution, preferably, the step of using the k-means algorithm to select the initial centroid and iteratively update it based on the user similarity in the user similarity noise matrix includes the following specific steps:
[0028] The maximum number of outliers in the user dataset is determined based on the preset outlier ratio of the user dataset.
[0029] The number of abnormal users identified by the box plot is compared with the number of the maximum outliers to determine whether there are still abnormal users in the user dataset.
[0030] If it exists, the average similarity avgSim between a user and other users is determined by the similarity-plus-noise matrix SimM, and the average similarity avgSim is used as the initial centroid. If it does not exist, the target user is selected as the initial centroid.
[0031] Using the furthest distance mode, the user with the lowest similarity is selected as the centroid during the iterative selection process;
[0032] Select the top q centroids with the highest similarity to the user, and take the centroid with the closest similarity as the centroid of the cluster to which the user belongs. Here, q is a preset value, and compared with the k value of the k-means algorithm, q < k.
[0033] When updating the centroid, calculate the similarity between each user in the cluster and all other users, and update the user with the highest sum of similarity as the new centroid.
[0034] In the above technical solution, preferably, the process of using the DBSCAN algorithm to filter out suspected abnormal users and using a box plot to determine the actual abnormal users includes:
[0035] The DBSCAN clustering algorithm is used to detect all suspected abnormal user sets in the user dataset, and the abnormality indicators in the suspected abnormal user sets are calculated.
[0036] Using the aforementioned abnormal indicators as input to the box plot, all abnormal indicators are detected, and the users corresponding to the abnormal indicators are identified as the actual abnormal users.
[0037] This invention also proposes a collaborative filtering recommendation system based on adaptive noise reduction and privacy protection, which applies the collaborative filtering recommendation method based on adaptive noise reduction and privacy protection disclosed in any of the above technical solutions, including:
[0038] The abnormal data filtering module is used to filter out suspected abnormal users from the user dataset using the DBSCAN algorithm and to identify the real abnormal users using a box plot, thereby filtering out abnormal user data from the user dataset.
[0039] An adaptive noise-adding module is used to calculate the similarity between the user datasets, and add noise of different sizes according to the similarity values to construct a user similarity noise-adding matrix;
[0040] The neighbor clustering selection module is used to select the initial centroid based on the user similarity in the user similarity-plus-noise matrix using the k-means algorithm and perform iterative updates to achieve user data clustering.
[0041] The project data filtering module is used to remove items already owned by the target user from the neighbor set formed by other users in the cluster where the target user is located, and obtain a recommendation list.
[0042] The project data recommendation module is used to calculate the rating of each project in the recommendation list and recommend the preset number of projects with the highest ratings to the target user.
[0043] In the above technical solution, preferably, the adaptive noise addition module adds noise of different magnitudes according to the similarity values between users in the following specific way:
[0044] Based on the similarity scores between users, users with higher similarity scores are assigned relatively more noise, while users with lower similarity scores are assigned relatively less noise.
[0045] In the above technical solution, preferably, the adaptive noise addition module is specifically used for:
[0046] For the similarity matrix Sim formed by the similarity between users, a privacy budget ε is preset, and user x is calculated. i With user x j The maximum privacy budget between them is [ε / (n-1)]. 2 ];
[0047] Search for user x in the similarity matrix Sim iThe minimum similarity between a user and another user is denoted as minSimilarity.
[0048] Calculate user x i With user x j The similarity noise between them is Noise' = Lap(Δf / ε'), where ε' = [ε / (n-1)]. 2 ]*[minSimilarity / Sim(d i ,d j )];
[0049] For user x i With user x j The similarity between Sim(d) i ,d j Adding noise, we get SimM(d) i ,d j ) = Sim(d i ,d j )+Noise';
[0050] Noise is added to the similarity between users to obtain the similarity-noise matrix SimM.
[0051] In the above technical solution, preferably, the neighbor clustering selection module is specifically used for:
[0052] The maximum number of outliers in the user dataset is determined based on the preset outlier ratio of the user dataset.
[0053] The number of abnormal users identified by the box plot is compared with the number of the maximum outliers to determine whether there are still abnormal users in the user dataset.
[0054] If it exists, the average similarity avgSim between a user and other users is determined by the similarity-plus-noise matrix SimM, and the average similarity avgSim is used as the initial centroid. If it does not exist, the target user is selected as the initial centroid.
[0055] Using the furthest distance mode, the user with the lowest similarity is selected as the centroid during the iterative selection process;
[0056] Select the top q centroids with the highest similarity to the user, and take the centroid with the closest similarity as the centroid of the cluster to which the user belongs. Here, q is a preset value, and compared with the k value of the k-means algorithm, q < k.
[0057] When updating the centroid, calculate the similarity between each user in the cluster and all other users, and update the user with the highest sum of similarity as the new centroid.
[0058] In the above technical solution, preferably, the abnormal data filtering module is specifically used for:
[0059] The DBSCAN clustering algorithm is used to detect all suspected abnormal user sets in the user dataset, and the abnormality indicators in the suspected abnormal user sets are calculated.
[0060] Using the aforementioned abnormal indicators as input to the box plot, all abnormal indicators are detected, and the users corresponding to the abnormal indicators are identified as the actual abnormal users.
[0061] Compared with existing technologies, the beneficial effects of this invention are as follows: By combining the DBSCAN algorithm and box plots for preprocessing the user dataset, abnormal user data is filtered out. By adding noise of different sizes to each user to construct a similarity noise matrix, the problem of blindly adding noise of the same size causing large deviations in cluster centroids can be avoided. The k-means algorithm is used for the selection of initial centroids and iterative updates of centroids. Finally, recommendation results are generated based on the cluster to which the target user belongs. This can, to a certain extent, avoid the accumulation of noise during the iteration process caused by adding noise in multiple places, thereby completing the protection of user privacy in the entire collaborative filtering recommendation algorithm and improving the accuracy of recommendation results. Attached Figure Description
[0062] Figure 1 This is a schematic diagram of the overall process of a collaborative filtering recommendation method based on adaptive noise reduction and privacy protection disclosed in an embodiment of the present invention;
[0063] Figure 2 This is a schematic diagram of a process for screening abnormal users using DBSCAN and box plots, as disclosed in an embodiment of the present invention.
[0064] Figure 3 This is a schematic diagram illustrating the construction process of a similarity noise matrix according to an embodiment of the present invention;
[0065] Figure 4 This is a schematic diagram of the process of user clustering using the k-means algorithm disclosed in one embodiment of the present invention;
[0066] Figure 5 This is a schematic diagram of a project recommendation process disclosed in one embodiment of the present invention. Detailed Implementation
[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0068] The present invention will now be described in further detail with reference to the accompanying drawings:
[0069] like Figure 1 As shown, a collaborative filtering recommendation method based on adaptive noise reduction and privacy protection provided by the present invention includes:
[0070] For the user dataset, the DBSCAN algorithm is used to filter out suspected abnormal users, and the box plot is used to identify the real abnormal users, thus removing abnormal user data from the user dataset.
[0071] Calculate the similarity between user datasets, and add noise of different sizes according to the similarity values. Add larger noise for high similarity and smaller noise for low similarity to construct a user similarity noise matrix;
[0072] Using the k-means algorithm, initial centroids are selected based on user similarity in the user similarity-plus-noise matrix, and iterative updates are performed to achieve user data clustering;
[0073] For the neighbor set formed by other users in the cluster where the target user is located, the items that the target user already owns are deleted from the neighbor set to obtain the recommendation list;
[0074] Calculate the rating for each item in the recommendation list, and recommend the preset number of items with the highest ratings to the target user.
[0075] In this implementation, the user dataset is preprocessed by combining the DBSCAN algorithm and box plots to filter out abnormal user data. By adding noise of different sizes to each user to construct a similarity noise matrix, the problem of blindly adding noise of the same size causing large deviations in cluster centroids can be avoided. The k-means algorithm is used to select the initial centroids and iteratively update them. Finally, recommendation results are generated based on the cluster to which the target user belongs. This approach can, to a certain extent, avoid the accumulation of noise during the iteration process caused by adding noise in multiple places, thereby completing the user privacy protection in the entire collaborative filtering recommendation algorithm and improving the accuracy of the recommendation results.
[0076] Specifically, the DBSCAN clustering algorithm is combined with box plots to remove outliers from the initial user set. This eliminates most of the noise for subsequent user clustering using the k-means algorithm, improving the accuracy of the recommendation results. Adding noise of varying magnitudes during user similarity calculations avoids the problem of blindly adding noise of the same magnitude, which can cause significant deviations in cluster centroids.
[0077] The implementation process mainly includes the following steps:
[0078] 1) Data preprocessing: Preprocessing the raw data, including data standardization and outlier removal;
[0079] 2) Similarity calculation: Identifying similar users by calculating the similarity between them;
[0080] 3) User privacy protection: User characteristics are protected for privacy through adaptive noise reduction technology;
[0081] 4) Neighbor selection: Based on the similarity calculation results, select the neighboring users who are most similar to the target user, and use the neighboring users as a reference for recommendations;
[0082] 5) Project Recommendation: Based on the historical project data of neighboring users, predict the target user's preference for unrated projects.
[0083] During implementation, similarity calculation includes, but is not limited to, the following methods:
[0084] (1) Calculate the similarity between users in the user dataset using the modified cosine similarity;
[0085] (2) Calculate the similarity between users in the user dataset using the Jaccard correlation coefficient;
[0086] (3) Calculate the similarity between users in the user dataset using rating habit similarity.
[0087] like Figure 2 As shown, in the above embodiment, preferably, the DBSCAN algorithm is used to filter out suspected abnormal users, and a box plot is used to determine the actual abnormal users. The specific process includes:
[0088] The DBSCAN clustering algorithm is used to detect all suspected abnormal user sets in the user dataset, and the abnormality indicators in the suspected abnormal user sets are calculated.
[0089] Using abnormal metrics as input to the box plot, all abnormal metrics are detected, and the users corresponding to the abnormal metrics are identified as the real abnormal users.
[0090] Specifically, the k-means algorithm is sensitive to outliers, and the accuracy of the algorithm is affected by the interference of outliers; while the DBSCAN algorithm is not sensitive to outliers in the dataset and can be used to filter out possible outliers in the dataset.
[0091] When screening outliers, it is necessary to know the number of outliers, but in general, the number of outliers is unknown. Box plots can solve this problem. Based on the output of the DBSCAN algorithm, users in clusters other than the target user's cluster are considered outliers. The silhouette coefficient of each user sample point is used as the outlier index. The outlier index is used as the input of the box plot. The box plot outputs the outlier index of the outlier. Then, the user corresponding to the outlier index of the outlier is considered the real outlier.
[0092] In the above implementation, preferably, the specific method of adding noise of different magnitudes according to the similarity values between users is as follows:
[0093] Based on the similarity scores between users, users with higher similarity scores are assigned relatively more noise, while users with lower similarity scores are assigned relatively less noise.
[0094] like Figure 3 As shown, specifically, the process of constructing the user similarity noise matrix includes:
[0095] For the similarity matrix Sim formed by the similarity between users, a privacy budget ε is preset. The total privacy budget for a user is ε / (n-1), and user x is calculated. i With user x j The maximum privacy budget between them is [ε / (n-1)]. 2 ];
[0096] Find user x in the similarity matrix Sim i With other users {x1,x2,…,x i-1 ,x i+1 ,…,x n The minimum similarity between} is denoted as minSimilarity;
[0097] Calculate user x i With user x j The similarity noise between them is Noise' = Lap(Δf / ε'), where ε' = [ε / (n-1)]. 2 ]*[minSimilarity / Sim(d i ,d j )], Δf is the sensitivity (as defined by the Laplace mechanism, given a dataset D, there exists a function f: D->R dIf its sensitivity is Δf, then the random algorithm M(D) = f(D) + Y provides ε-differential privacy protection, where Y ~ Lap(Δf / ε) is random noise that follows a Laplace distribution with a scale parameter of Δf / ε.
[0098] For user x i With user x j The similarity between Sim(d) i ,d j Adding noise, we get SimM(d) i ,d j ) = Sim(d i ,d j )+Noise';
[0099] Noise is added to the similarity between users to obtain the similarity-noise matrix SimM.
[0100] like Figure 4 As shown, in the above implementation, preferably, the k-means algorithm is used to select the initial centroid based on the user similarity in the user similarity-plus-noise matrix and perform iterative updates. The specific process includes:
[0101] The maximum number of outliers in the user dataset is determined based on the pre-defined outlier ratio of the user dataset.
[0102] The number of anomalous users identified by the box plot is compared with the number of the largest outliers to determine whether there are still anomalous users in the user dataset.
[0103] If it exists, the average similarity avgSim between a user and other users is determined by the similarity-plus-noise matrix SimM, and the average similarity avgSim is used as the initial centroid. If it does not exist, the target user is selected as the initial centroid.
[0104] Using the farthest distance mode, the user with the smallest similarity is selected as centroid u1 during the iterative selection of centroids. That is, when selecting the second initial centroid u2, the user with the smallest similarity to u1 should be selected as the target point according to the similarity matrix SimM. When selecting the third target point, the user with the smallest similarity to u1 and u2 is selected as the target point according to the similarity matrix SimM. And so on, to obtain the k centroids required by the k-means algorithm.
[0105] Select the top q centroids with the highest similarity to the user, and take the centroid with the closest similarity as the centroid of the cluster to which the user belongs. Here, q is a preset value, and compared with the k value of the k-means algorithm, q < k.
[0106] When updating the centroid, calculate the similarity between each user in the cluster and all other users, and update the user with the highest sum of similarity as the new centroid.
[0107] In this implementation, the initial centroid is selected based on the number of excluded abnormal users. This approach can, on the one hand, avoid interference from abnormal users to a certain extent, and on the other hand, ensure that the initial centroids are distributed in different clusters as much as possible to improve the clustering speed. When updating the cluster centroids, the KNN idea is introduced, which can, to a certain extent, avoid the problem of noise being added in multiple places and accumulating continuously during the iteration process, resulting in large differences between similar users in the same cluster as the target user.
[0108] like Figure 5 As shown, for user data clustering obtained by the k-means algorithm, the output is the neighbor set formed by all users in the cluster of the target user, and the list P of existing items in the neighbor user set is found. neib In P neib Remove items already owned by the target user u from the list and construct a recommendation list P. sel Calculate P sel The score for each item in P. sel Select the n highest-rated items and recommend them to the target user u.
[0109] In this process, the methods for calculating the score for each item mainly include the following:
[0110] (1) Use weights to calculate the similarity between legally calculated items;
[0111] (2) Calculate the similarity between items using the weighted average method.
[0112] This invention also proposes a collaborative filtering recommendation system based on adaptive noise reduction and privacy protection, which applies the collaborative filtering recommendation method based on adaptive noise reduction and privacy protection disclosed in any of the above embodiments, including:
[0113] The abnormal data filtering module is used to filter out suspected abnormal users from the user dataset using the DBSCAN algorithm and to identify the real abnormal users using box plots, thereby filtering out abnormal user data from the user dataset.
[0114] The adaptive noise addition module is used to calculate the similarity between user datasets and add noise of different sizes according to the similarity value. Larger noise is added for high similarity and smaller noise is added for low similarity, thus constructing a user similarity noise addition matrix.
[0115] The neighbor clustering selection module is used to select the initial centroid based on the user similarity in the user similarity-plus-noise matrix using the k-means algorithm and iteratively update it to achieve user data clustering.
[0116] The project data filtering module is used to remove items already owned by the target user from the neighbor set formed by other users in the cluster where the target user is located, and obtain a recommendation list.
[0117] The project data recommendation module is used to calculate the rating of each project in the recommendation list and recommend the preset number of projects with the highest ratings to the target user.
[0118] In the above embodiments, preferably, the adaptive noise addition module adds noise of different magnitudes according to the similarity values between users in the following specific way:
[0119] Based on the similarity scores between users, users with higher similarity scores are assigned relatively more noise, while users with lower similarity scores are assigned relatively less noise.
[0120] In the above embodiments, preferably, the adaptive noise addition module is specifically used for:
[0121] For the similarity matrix Sim formed by the similarity between users, a privacy budget ε is preset, and user x is calculated. i With user x j The maximum privacy budget between them is [ε / (n-1)]. 2 ];
[0122] Find user x in the similarity matrix Sim i The minimum similarity between a user and another user is denoted as minSimilarity.
[0123] Calculate user x i With user x j The similarity noise between them is Noise' = Lap(Δf / ε'), where ε' = [ε / (n-1)]. 2 ]*[minSimilarity / Sim(d i ,d j )];
[0124] For user x i With user x j The similarity between Sim(d) i ,d j Adding noise, we get SimM(d) i ,d j ) = Sim(d i ,d j )+Noise';
[0125] Noise is added to the similarity between users to obtain the similarity-noise matrix SimM.
[0126] In the above embodiments, preferably, the neighbor clustering selection module is specifically used for:
[0127] The maximum number of outliers in the user dataset is determined based on the pre-defined outlier ratio of the user dataset.
[0128] The number of anomalous users identified by the box plot is compared with the number of the largest outliers to determine whether there are still anomalous users in the user dataset.
[0129] If it exists, the average similarity avgSim between a user and other users is determined by the similarity-plus-noise matrix SimM, and the average similarity avgSim is used as the initial centroid. If it does not exist, the target user is selected as the initial centroid.
[0130] Using the furthest distance mode, the user with the lowest similarity is selected as the centroid during the iterative selection process;
[0131] Select the top q centroids with the highest similarity to the user, and take the centroid with the closest similarity as the centroid of the cluster to which the user belongs. Here, q is a preset value, and compared with the k value of the k-means algorithm, q < k.
[0132] When updating the centroid, calculate the similarity between each user in the cluster and all other users, and update the user with the highest sum of similarity as the new centroid.
[0133] In the above embodiments, preferably, the abnormal data filtering module is specifically used for:
[0134] The DBSCAN clustering algorithm is used to detect all suspected abnormal user sets in the user dataset, and the abnormality indicators in the suspected abnormal user sets are calculated.
[0135] Using abnormal metrics as input to the box plot, all abnormal metrics are detected, and the users corresponding to the abnormal metrics are identified as the real abnormal users.
[0136] The collaborative filtering recommendation system based on adaptive noise reduction and privacy protection disclosed in the above embodiments has modules whose functions correspond to the steps in the collaborative filtering recommendation method based on adaptive noise reduction and privacy protection disclosed in the above embodiments. In the implementation process, the implementation shall be carried out with reference to the description of the above embodiments, and will not be repeated here.
[0137] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A collaborative filtering recommendation method based on adaptive noise reduction and privacy protection, characterized in that, include: For the user dataset, the DBSCAN algorithm is used to filter out suspected abnormal users, and the box plot is used to identify the real abnormal users, thereby removing the abnormal user data from the user dataset. Calculate the similarity between the user datasets, and add noise of different sizes according to the similarity values to construct a user similarity noise matrix; Using the k-means algorithm, initial centroids are selected based on user similarity in the user similarity-plus-noise matrix, and iterative updates are performed to achieve user data clustering; For the neighbor set formed by other users in the cluster to which the target user belongs, the items that the target user already owns are deleted from the neighbor set to obtain the recommendation list; Calculate the rating for each item in the recommendation list, and recommend a preset number of items with the highest ratings to the target user; The specific method for adding noise of different magnitudes according to the similarity values between users is as follows: Based on the similarity scores between users, users with higher similarity scores are given relatively more noise, while users with lower similarity scores are given relatively less noise. The process of adding noise of different magnitudes to each user's similarity value to construct a user similarity noise matrix includes: For the similarity matrix Sim formed by the similarity between users, a privacy budget ε is preset, and the user's privacy budget is calculated. x i With users x j The maximum privacy budget between them is [ε / (n-1)]. 2 ]; Searching for users in the similarity matrix Sim x i The minimum similarity between a user and another user is denoted as minSimilarity. Calculate users x i With users x j The similarity noise between them is Noise' = Lap(Δf / ε'), where ε' = [ε / (n-1)]. 2 ]*[minSimilarity / Sim( d i , d j )]; For users x i With users x j Similarity between Sim( d i , d j Adding noise, we get SimM( d i , d j )= Sim( d i , d j )+ Noise'; Noise is added to the similarity between users to obtain the similarity-noise matrix SimM.
2. The collaborative filtering recommendation method based on adaptive noise reduction and privacy protection according to claim 1, characterized in that, The process of using the k-means algorithm to select the initial centroid and iteratively update it based on the user similarity in the user similarity-plus-noise matrix includes: The maximum number of outliers in the user dataset is determined based on the preset outlier ratio of the user dataset. The number of abnormal users identified by the box plot is compared with the number of the maximum outliers to determine whether there are still abnormal users in the user dataset. If it exists, the average similarity avgSim between a user and other users is determined by the similarity-plus-noise matrix SimM, and the average similarity avgSim is used as the initial centroid. If it does not exist, the target user is selected as the initial centroid. Using the furthest distance mode, the user with the lowest similarity is selected as the centroid during the iterative selection process; Select the top q centroids with the highest similarity to the user, and take the centroid with the closest similarity as the centroid of the cluster to which the user belongs. Here, q is a preset value, and compared with the k value of the k-means algorithm, q < k. When updating the centroid, calculate the similarity between each user in the cluster and all other users, and update the user with the highest sum of similarity as the new centroid.
3. The collaborative filtering recommendation method based on adaptive noise reduction and privacy protection according to claim 2, characterized in that, The process of using the DBSCAN algorithm to filter out suspected anomalous users and using box plots to identify the actual anomalous users includes: The DBSCAN clustering algorithm is used to detect all suspected abnormal user sets in the user dataset, and the abnormality indicators in the suspected abnormal user sets are calculated. Using the aforementioned abnormal indicators as input to the box plot, all abnormal indicators are detected, and the users corresponding to the abnormal indicators are identified as the actual abnormal users.
4. A collaborative filtering recommendation system based on adaptive noise reduction and privacy protection, characterized in that, The collaborative filtering recommendation method based on adaptive noise reduction and privacy protection as described in any one of claims 1 to 3 includes: The abnormal data filtering module is used to filter out suspected abnormal users from the user dataset using the DBSCAN algorithm and to identify the real abnormal users using a box plot, thereby filtering out abnormal user data from the user dataset. An adaptive noise-adding module is used to calculate the similarity between the user datasets, and add noise of different sizes according to the similarity values to construct a user similarity noise-adding matrix; The neighbor clustering selection module is used to select the initial centroid based on the user similarity in the user similarity-plus-noise matrix using the k-means algorithm and perform iterative updates to achieve user data clustering. The project data filtering module is used to remove items already owned by the target user from the neighbor set formed by other users in the cluster where the target user is located, and obtain a recommendation list. The project data recommendation module is used to calculate the rating of each project in the recommendation list and recommend the preset number of projects with the highest ratings to the target user.
5. The collaborative filtering recommendation system based on adaptive noise reduction and privacy protection according to claim 4, characterized in that, The neighbor clustering selection module is specifically used for: The maximum number of outliers in the user dataset is determined based on the preset outlier ratio of the user dataset. The number of abnormal users identified by the box plot is compared with the number of the maximum outliers to determine whether there are still abnormal users in the user dataset. If it exists, the average similarity avgSim between a user and other users is determined by the similarity-plus-noise matrix SimM, and the average similarity avgSim is used as the initial centroid. If it does not exist, the target user is selected as the initial centroid. Using the furthest distance mode, the user with the lowest similarity is selected as the centroid during the iterative selection process; Select the top q centroids with the highest similarity to the user, and take the centroid with the closest similarity as the centroid of the cluster to which the user belongs. Here, q is a preset value, and compared with the k value of the k-means algorithm, q < k. When updating the centroid, calculate the similarity between each user in the cluster and all other users, and update the user with the highest sum of similarity as the new centroid.
6. The collaborative filtering recommendation system based on adaptive noise reduction and privacy protection according to claim 5, characterized in that, The abnormal data filtering module is specifically used for: The DBSCAN clustering algorithm is used to detect all suspected abnormal user sets in the user dataset, and the abnormality indicators in the suspected abnormal user sets are calculated. Using the aforementioned abnormal indicators as input to the box plot, all abnormal indicators are detected, and the users corresponding to the abnormal indicators are identified as the actual abnormal users.