A matrix factorization recommendation method based on shuffle differential privacy

Abstract

A matrix factorization recommendation method based on shuffling differential privacy includes: acquiring a dataset of interactions between users and items, inputting it into a matrix factorization recommendation model for processing to obtain item recommendation results for users; training the matrix factorization recommendation model: preprocessing model information on the server side; calculating user feature matrices on the user side based on the preprocessed model information, and generating a message set by locally perturbing each message, then uploading the generated message set to a shuffling server; the shuffling server receiving the locally perturbed messages from the user side, and performing a shuffling operation on these messages using a differential privacy shuffling model to obtain a shuffled message dataset, which is then transmitted to the server side; aggregating and traversing the shuffled message dataset, then updating the item feature matrix information and user feature matrix information; performing privacy budget segmentation, and finally integrating the user feature matrix and item feature matrix to construct the matrix factorization recommendation model.

Description

Technical Field

[0001] This invention relates to the field of differential privacy technology, and more specifically, to a matrix factorization recommendation method based on shuffle differential privacy. Background Technology

[0002] With the widespread adoption of mobile devices, consumers are increasingly relying on online marketplaces for shopping. Faced with a wealth of choices, the decision-making process has become more complex, making recommendation systems increasingly important. Recommendation systems not only help consumers discover products of interest but also provide personalized suggestions. By collecting and analyzing user behavior data, recommendation systems can optimize the user experience. However, this data often contains sensitive personal information, such as user rating patterns or item selection preferences, which may lead to the leakage of users' privacy information, such as gender and political affiliation. Therefore, protecting user privacy has become a critical issue in recommendation systems.

[0003] Among numerous privacy protection technologies, Differential Privacy (DP) has become one of the core standards in the field of data protection. Traditional privacy protection methods are mostly based on Centralized Differential Privacy (DP), which assumes the existence of a trusted server that adds noise to the aggregated data. However, in practical applications, recommendation service providers may be untrustworthy, and the leakage of user data stored centrally could have serious consequences. To avoid this risk, Local Differential Privacy (Local DP) has been proposed. In this approach, users perturb the data locally before uploading it, thereby avoiding the risk of data leakage from centralized storage. However, Local Differential Privacy typically requires a high amount of noise, which can lead to a significant decrease in model accuracy, and in scenarios with multiple data interactions, the privacy budget is consumed too quickly, limiting its application in big data analytics and machine learning.

[0004] To address this challenge, the shuffle model of DP has been proposed as a promising alternative. In the shuffle model, the shuffler randomly rearranges user information before publishing it to the server. Since the shuffler only rearranges the information without accessing its content, it is considered semi-trusted and can be implemented via anonymous channels or edge servers. Through shuffling, users can inject less noise (low local privacy protection), thus significantly amplifying the privacy protection effect. Recent research shows that, from the server's perspective, shuffled local ε-DP messages from n users can achieve... Therefore, the utility of the shuffle model exceeds that of the local model and may approach that of the central model.

[0005] On the other hand, traditional content-based recommendation systems rely on collecting large amounts of users' personal information and interaction data to provide personalized suggestions. This not only increases the risk of privacy breaches but also raises the demand for computing resources. In contrast, collaborative filtering recommendation systems focus more on users' evaluations of products and behavioral patterns, and can better adapt to the increase of new users and new products. In particular, model-based collaborative filtering methods can make recommendations by learning the latent features of users and products, and have strong generalization ability. However, applying Local Differential Privacy (LDP) to matrix factorization faces two major challenges: first, the item dimension (usually tens of thousands to millions) causes gradient perturbation noise to increase linearly with the increase of dimension, which seriously affects the model's utility; second, the round-by-round allocation of the privacy budget during iterative optimization (e.g., when the total budget is ϵ, only ϵ / k is allocated in each round) causes the noise to amplify exponentially with the number of iterations, making it difficult for the model to converge.

[0006] In summary, existing technologies still face the following problems and challenges: 1. Low data utility of traditional difference matrix factorization (DMF) recommendation systems: Because the matrix factorization process involves multiple iterative optimizations, the privacy budget needs to be partitioned. If the number of iterations is much greater than the privacy budget, the privacy budget in a single round will approach zero, causing the system to be unable to obtain effective statistical information from noise, resulting in low data utility. 2. High communication and computational overhead: DMF recommendation systems require multiple interactions between the user and the server, and the user also needs to perform complex calculations to meet differential privacy requirements. Due to privacy protection, the server needs to download the complete item feature matrix during communication with the user, increasing communication and computational overhead.

[0007] To address these issues, this invention proposes a privacy-preserving matrix factorization framework that combines dimensionality reduction techniques and a multi-message shuffling model. By using random projection to map high-dimensional gradients to a low-dimensional space, sensitivity to noise is significantly reduced. Furthermore, a sampling-based binary perturbation mechanism only perturbs the randomly selected gradient dimension at the user end, thereby reducing communication overhead. In addition, leveraging the anonymization properties of the shuffling model and incorporating privacy amplification effect analysis methods, a tighter theoretical privacy boundary is provided for the privacy protection mechanism, increasing the privacy budget for each iteration and further enhancing the utility of the recommendation system. Summary of the Invention

[0008] The present invention aims to overcome at least one of the defects (deficiencies) of the prior art and provide a matrix factorization recommendation method based on shuffle differential privacy, which solves the problems of low data utility and high communication and computational overhead of traditional differential matrix factorization recommendation systems.

[0009] The technical solution adopted in this invention is a matrix factorization recommendation method based on shuffle differential privacy. The method is as follows: obtain the interaction dataset between users and items, and input it into the matrix factorization recommendation model for processing, so as to obtain the item recommendation results for users;

[0010] It also includes training the matrix factorization recommendation model, the training steps of which are as follows:

[0011] S1: Preprocess model information on the server side, including setting system parameters, initializing system information, and setting optimization goals;

[0012] S2: Calculate the user feature matrix on the user end based on the preprocessed model information, and generate a message set by locally perturbing each message. Then, upload the generated message set to the shuffling server.

[0013] S3: The shuffling server receives messages from the user end after local perturbation, and uses a differential privacy shuffling model to shuffle these messages to obtain a shuffled message dataset, which is then transmitted to the server.

[0014] S4: Aggregate and traverse the shuffled message dataset, then update the project feature matrix information on the server side and the user feature matrix information on the user side;

[0015] S5: Perform privacy budget segmentation to ensure that differential privacy constraints are met in each iteration of the model. Finally, integrate the user feature matrix and the item feature matrix to construct a matrix factorization recommendation model.

[0016] This application significantly improves the privacy protection capability of matrix factorization recommendation by employing a multi-message shuffling matrix factorization recommendation mechanism, while ensuring the usability of the method. It effectively solves the trade-off between privacy protection and model performance in existing technologies, and reduces communication overhead during model iteration. Each user only needs to select two dimensions of single-bit data from the calculated item recommendation matrix for perturbation and transmission, without uploading the entire gradient, thus reducing computational and communication overhead on the user side. Simultaneously, the server uses a randomized dimensionality reduction matrix to project the feature gradient matrix into a low-rank space, allowing users to update feature vectors by downloading a lower-rank update matrix with fewer parameters, further reducing communication overhead between users and the server. Furthermore, this application can also ensure privacy protection. By introducing a shuffle differential privacy model, users calculate and randomly sample the update information of the item matrix. The data is perturbed by adding a small amount of noise locally. After being anonymized by a multi-message shuffle protocol on the shuffle server, the perturbation gradient information of the sampling dimension is reported. The process of each user randomly selecting the data dimension is independent of each other. The shuffle process severs the correlation between the data and the original user by randomly changing the order of user messages. This makes the global privacy protection level from the server's perspective significantly higher than the local privacy budget of a single user, thereby achieving high-strength privacy protection under low local noise conditions.

[0017] Step S2 includes:

[0018] S21: Each user uses their local preference rating to calculate the recommendation feature matrix and update the gradient;

[0019] S22: Perform random sampling to constrain the gradient values ​​within the desired data domain;

[0020] S23: Locally scramble the sampled data points;

[0021] S24: Generate a message set by splitting the perturbed result into independent messages and then adding them to the message set. If the size of the message set is insufficient to handle the number of messages generated by each user... Then repeat steps S22 and S23 until the size of the message set equals the number of messages generated by each user. ;

[0022] S25: Upload the generated message set to the shuffling server.

[0023] This application employs gradient updates, random sampling, local perturbation, message generation, and uploading to a shuffling server to ensure that each uploaded message has undergone local perturbation during the recommendation system optimization process, eliminating the presence of original data and effectively protecting user data privacy. Simultaneously, it maintains the system's efficiency and accuracy, enabling the recommendation system to provide personalized recommendations while avoiding privacy leaks, thus achieving a good balance between privacy and performance.

[0024] Preferably, step S21 includes:

[0025] S211: For each user Existing projects Set a marker variable for the rating record. The value is 1 if it is not 1, otherwise it is 0.

[0026] S212: Calculate the gradient vector using local rating record data, and calculate the vector in the low-rank space using a randomized dimensionality reduction matrix.

[0027] The method described in this application effectively reduces communication and computational overhead, improves the training efficiency and prediction accuracy of the recommendation system, reduces the risk of overfitting, and can handle large-scale datasets. In particular, the introduction of low-rank space allows users to calculate gradient information using local preference data and convert the gradient information into sparse triplet form. The user then uploads a set of pruned and perturbed sparse triplets. The server selects a dimension parameter much smaller than the number of recommended items and generates a random dimension reduction matrix based on this dimension parameter to reduce the dimensionality of the recommended item matrix. This reduces the computational and communication overhead between the user and the server, enabling the system to significantly improve computational efficiency and model generalization ability when dealing with large-scale sparse matrices.

[0028] More preferably, in step S212, the gradient vector calculation formula is:

[0029]

[0030] in, Represents the gradient vector; Indicates a marker variable; Represents the user feature vector; Represents each user Having the project Preference rating; express Transpose of; Represents the feature vector of the project; Indicates the number of recommended items;

[0031] The formula for calculating the vector in the low-rank space is:

[0032]

[0033] in, Represents a low-rank space vector; This represents a randomized reduced-dimensional matrix.

[0034] Preferably, step S22 includes:

[0035] S221: The user samples two dimensional variables from a uniform distribution. and ,in, The sampling set is , The sampling set is ,in Represents the dimension of a low-rank spatial vector;

[0036] S222: Let and The low-rank gradient values ​​corresponding to the two latitudes are ,like The value is outside the data field If so, the data needs to be pruned to constrain the gradient value within the desired data domain.

[0037] By using uniform sampling and pruning, the model can avoid overfitting to a certain dimension during the learning process, while also ensuring that the gradient values ​​updated during training are reasonable. This avoids overfitting, ensures that the gradient values ​​remain within a reasonable range, reduces large changes in computation and storage, and improves the efficiency of the training process.

[0038] Preferably, in step S23, the local perturbation of the sampled data points includes:

[0039] In the Bernoulli distribution, the variable is identified from the sampling sign. ,if A value of 1 returns the perturbation gradient. Otherwise return ,in Represents the dimension of the implicit vector data; Privacy parameters indicating user disruption mechanisms; This represents the total number of iterations of the algorithm.

[0040] Through this perturbation mechanism, the model can introduce randomness during gradient updates, increase the diversity of the training process, avoid getting trapped in local optima, and thus enhance the model's generalization ability and robustness. The adaptive perturbation strength and directionality make this mechanism more efficient in the gradient optimization process, which helps to improve the model's performance in complex tasks.

[0041] Preferably, step S3 includes:

[0042] S31: Summarize the noise data in all user-submitted disturbance messages and integrate them into a temporary dataset according to a unified format, eliminating user labeling information;

[0043] S32: Perform a reverse traversal random permutation on the temporary dataset. Starting from the last data, randomly select a data from a previous position each time and swap it with the current position until the traversal is complete, ensuring that all data positions are uniformly and randomly shuffled.

[0044] S33: Transmit the obfuscated message dataset to the server.

[0045] This invention effectively reduces privacy budget consumption. By utilizing a shuffle differential privacy mechanism combined with a fully random tree structure, and through random permutation operations via reverse traversal, the data order is uniformly shuffled, avoiding correlation between data and avoiding dynamic splitting rule calculations, thus significantly reducing privacy budget consumption. At the same time, it fully utilizes the privacy enhancement effect of the shuffle protocol, effectively enhancing data security and privacy, while avoiding potential leakage risks and improving the system's privacy protection capabilities.

[0046] Preferably, in step S4, the aggregated and shuffled message dataset includes: a global message set shuffled by the server, mean calculation of the perturbation gradient information for each data dimension, and reconstruction of the low-rank gradient matrix on the server side. For each latitude point The statistics include the gradient set at that latitude, and the gradient set... ,in, This represents the obfuscated message dataset; Indicates user The generated message set; This represents the set union operation; express The perturbation gradient value at the location; Represents a set Index information in the elements of a triplet;

[0047] Next, the server calculates the mean of the messages for each data dimension, where the dimension point... The element value is , express The gradient value in; This represents the set of gradients.

[0048] This step optimizes the processing and updating of gradient information by reconstructing the low-rank gradient matrix on the server side and averaging the perturbation gradients for each data dimension. Specifically, by shuffling the global message set, statistically analyzing the gradient set for each data dimension, and averaging the results, it ensures that true gradient information can be effectively extracted when dealing with large amounts of perturbation data. This reduces redundancy and interference in the data, thereby improving the training efficiency and stability of the model, ensuring the reliability and accuracy of gradient information, enhancing the stability and precision of model training, reducing computational complexity and storage requirements, enhancing the model's generalization ability, and ensuring a more efficient and accurate training process.

[0049] Preferably, in step S4, updating the project feature matrix information on the server side includes: the server first uses a pseudo-inverse random dimensionality reduction matrix. Reconstruct the project gradient matrix ,in Then, the updated project feature matrix is ​​calculated using the reconstructed project gradient matrix. The calculation formula is as follows:

[0050]

[0051] in, Represents the project feature matrix; This refers to the learning rate in this update. This represents the penalty parameter for the weights of the project's feature vectors.

[0052] Preferably, in step S4, updating the user feature matrix information on the user end includes: user Download the reconstructed low-rank gradient matrix The project gradient matrix is ​​reconstructed locally, and then the updated gradient of the user feature vector is calculated using the following formula:

[0053]

[0054] in, Represents user feature vector The update gradient; Indicates a marker variable; Represents the feature vector of the project; Represents each user Having the project Preference rating; Represents user feature vector Transpose of; Indicates the number of recommended items;

[0055] And by using this gradient information to update the user feature vector, we can obtain...

[0056]

[0057] in, This represents the updated user feature vector; This refers to the learning rate in this update. This represents the penalty parameter for the weights of the user feature vector.

[0058] This step reconstructs the project gradient matrix on the server side using a pseudo-inverse random dimensionality reduction matrix, projects the feature gradient matrix into a low-rank space, and updates the project feature matrix using the reconstructed matrix. This improves the efficiency and accuracy of updating the project feature matrix, allowing users to complete the feature vector update operation simply by downloading a low-rank update matrix with fewer parameters, further reducing the communication overhead between the user and the server. Simultaneously, the user client optimizes the calculation and update process of the user feature matrix by downloading the reconstructed low-rank gradient matrix and performing local updates, effectively reducing data transmission volume, lowering computational and communication overhead on the user side, and maintaining good computational stability and convergence in a distributed environment. This improves the model's generalization ability, avoids overfitting, reduces computational overhead, and enhances the overall system performance.

[0059] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0060] (1) Reduce privacy budget consumption: This invention utilizes the shuffle differential privacy mechanism and combines it with a fully random tree structure to avoid dynamic splitting rule calculation, thereby significantly reducing privacy budget consumption, while making full use of the privacy enhancement effect of the shuffle protocol.

[0061] (2) Reduced communication and computational overhead: Users calculate gradient information using local preference data and convert the gradient information into sparse triples, then upload a set of pruned and perturbed sparse triples. The server selects a dimension parameter much smaller than the number of recommended items and generates a random dimension reduction matrix based on this parameter to reduce the dimensionality of the recommended item matrix, thereby reducing the computational and communication overhead between the user and the server. Attached Figure Description

[0062] Figure 1 The flowchart of the model training method provided in this embodiment is shown.

[0063] Figure 2 This is a schematic diagram of the experimental test results provided in this embodiment. Detailed Implementation

[0064] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the invention. To better illustrate the following embodiments, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions; it is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0065] Example 1

[0066] The technical solution adopted in this embodiment is a matrix factorization recommendation method based on shuffle differential privacy. The method is as follows: obtain the interaction dataset between users and items, and input it into the matrix factorization recommendation model for processing, so as to obtain the item recommendation results for users;

[0067] Preferably, the method further includes training the matrix factorization recommendation model, such as... Figure 1 As shown, the training steps are as follows:

[0068] Step S1: Preprocess model information on the server side, including setting system parameters, initializing system information, and setting optimization goals;

[0069] Preferably, setting system parameters includes: setting system parameters. The amount of user data in the recommendation system is... The number of recommended projects is The number of messages generated by each user is The privacy parameters for the user disruption mechanism are: The dimensions of the latent vector data are (generally The dimension of the low-rank space vector is (generally The random dimensionality reduction matrix is The user feature matrix is The project feature matrix is .

[0070] The initialization system information includes: assuming that each element in the random dimensionality reduction matrix has a mean of 0 and a standard deviation of . The distribution is Gaussian. Simultaneously, all parameters in the user feature matrix and project feature matrix are initialized to 0.

[0071] The optimization goals include: setting each user... Having the project Preference rating The user feature vector is The project feature vector is The specific optimization objective of the matrix factorization recommendation model is as follows:

[0072]

[0073] in, A set of all user preference ratings for the project. For set The number of elements included and These are the penalty parameters for the user feature vector and the project feature vector, respectively, used to prevent overfitting during model training.

[0074] Step S2: Calculate the user feature matrix on the user end based on the preprocessed model information, and perform local perturbation on each message to generate a message set. Then, upload the generated message set to the shuffling server.

[0075] Preferably, step S2 includes the following steps:

[0076] Step S21: In order to optimize the objective function Each user uses their local preference rating to calculate the recommendation feature matrix and update the gradient;

[0077] Preferably, step S21 includes:

[0078] Step S211: For each user Existing projects Set a marker variable for the rating record. The value is 1 if it is not 1, otherwise it is 0.

[0079] Step S212: Calculate the gradient vector using local rating record data, and calculate the vector in the low-rank space using a randomized reduced-dimensional matrix.

[0080] More preferably, in step S212, the gradient vector calculation formula is:

[0081]

[0082] in, Represents the gradient vector; Indicates a marker variable; Represents the user feature vector; Represents each user Having the project Preference rating; express Transpose of; Represents the feature vector of the project; Indicates the number of recommended items;

[0083] The formula for calculating the vector in the low-rank space is:

[0084]

[0085] in, Represents a low-rank space vector; This represents a randomized reduced-dimensional matrix.

[0086] Thus, in this embodiment, the above operations effectively reduce communication and computational overhead, improve the training efficiency and prediction accuracy of the recommendation system, reduce the risk of overfitting, and enable the handling of large-scale datasets. In particular, the introduction of low-rank space allows users to calculate gradient information using local preference data and convert the gradient information into sparse triplet form. The user then uploads a set of pruned and perturbed sparse triplets. The server selects a dimension parameter much smaller than the number of recommended items and generates a random dimension reduction matrix based on this dimension parameter to reduce the dimensionality of the recommended item matrix. This reduces the computational and communication overhead between the user and the server, enabling the system to significantly improve computational efficiency and model generalization ability when handling large-scale sparse matrices.

[0087] Step S22: Perform random sampling to constrain the gradient values ​​within the desired data domain;

[0088] Preferably, step S22 includes:

[0089] Step S221: To reduce communication costs, the user samples two dimensional variables from a uniform distribution. and ,in, The sampling set is , The sampling set is ,in Represents the dimension of a low-rank spatial vector;

[0090] Step S222: Let and The low-rank gradient values ​​corresponding to the two latitudes are ,like The value is outside the data field If so, the data needs to be pruned to constrain the gradient value within the desired data domain.

[0091] By uniform sampling and pruning, the model can avoid overfitting to a certain dimension during the learning process, while also ensuring that the updated gradient values ​​during training are reasonable, thus avoiding overfitting and keeping the gradient values ​​within a reasonable range. This reduces large changes in computation and storage, and improves the efficiency of the training process.

[0092] Step S23: Locally scramble the sampled data points;

[0093] Preferably, to protect privacy, users need to use a perturbation mechanism to clean the sampled data points. In step S23, the local perturbation of the sampled data points includes:

[0094] In the Bernoulli distribution, the variable is identified from the sampling sign. ,Right now ,if A value of 1 returns the perturbation gradient. Otherwise return ,in Represents the dimension of the implicit vector data; Privacy parameters indicating user disruption mechanisms; This represents the total number of iterations of the algorithm.

[0095] Thus, through this perturbation mechanism in this embodiment, the model can introduce randomness during gradient updates, increase the diversity of the training process, avoid getting trapped in local optima, thereby enhancing the model's generalization ability and robustness. The adaptive perturbation strength and directionality make this mechanism more efficient in the gradient optimization process, which helps to improve the model's performance in complex tasks.

[0096] Step S24: Generate message set To optimize the utility of the recommendation system, the gradient error rate is reduced by splitting the perturbed results into independent messages, where each message has the following format: Then, the generated messages are added to the message set. If the size of the message set is insufficient to handle the number of messages generated by each user... Then repeat steps S22 and S23 until the size of the message set meets the requirements;

[0097] S25: The user will generate a set of messages. Uploaded to the shuffling server. Each uploaded message has been locally perturbed and does not contain the original data.

[0098] Through operations such as gradient updates, random sampling, local perturbation, message generation, and uploading to the shuffling server, it is ensured that each uploaded message has been locally perturbed during the recommendation system optimization process, and does not contain the original data. User data privacy is effectively protected, while maintaining the system's efficiency and accuracy. This allows the recommendation system to provide personalized recommendations while avoiding privacy leaks when processing user data, ensuring a good balance between privacy and performance.

[0099] Step S3: The shuffling server receives messages from the user end after local perturbation. These messages have been locally noiseed. The server uses a differential privacy shuffling model to shuffle these messages to obtain a shuffled message dataset, which is then transmitted to the server.

[0100] Preferably, step S3 includes:

[0101] S31: Summarize the noise data in all user-submitted disturbance messages and integrate them into a temporary dataset according to a unified format, eliminating user labeling information;

[0102] S32: Perform a reverse traversal random permutation on the temporary dataset. Starting from the last data, randomly select a data from a previous position each time and swap it with the current position until the traversal is complete, ensuring that all data positions are uniformly and randomly shuffled.

[0103] S33: Obfuscate the message dataset The data is transmitted to the server, eliminating the permutation correlation between data, so that the server cannot infer the correspondence between data and users through features such as order and context, and can only mine based on overall statistical features.

[0104] This effectively reduces privacy budget consumption. By utilizing the shuffle differential privacy mechanism and combining it with a fully random tree structure, the data order is uniformly shuffled through random permutation operations in reverse traversal, avoiding correlation between data and avoiding dynamic splitting rule calculations, thus significantly reducing privacy budget consumption. At the same time, it fully utilizes the privacy enhancement effect of the shuffle protocol, effectively enhancing data security and privacy, avoiding potential leakage risks, and improving the system's privacy protection capabilities.

[0105] In privacy amplification analysis, based on dimensional sampling and mechanism characteristics, the privacy analysis of this machine is reduced to a privacy analysis problem of a stochastic response mechanism. Combined with privacy amplification effect analysis techniques, the privacy level of the matrix factorization recommendation system construction process is analyzed, and the privacy protection level is effectively enhanced. In this embodiment, the derived global privacy protection level after shuffling can reach [a certain level]. -DP's tightness limit (where For the number of users, for , (where is the desired privacy leakage risk parameter), compared to the traditional Local Differential Privacy (LDP) method, the global privacy leakage risk is significantly reduced under the same local budget. This theoretical result provides a rigorous guarantee for the balance between privacy and utility, ensuring that while protecting user privacy, it can still support large-scale data analysis and model building with high utility.

[0106] In the differential privacy shuffling model, communication overhead is reduced by employing dimensionality sampling and random projection dimensionality reduction techniques, and anonymized statistics are performed on the data using a multi-message shuffling protocol. The introduction of multi-messages effectively mitigates data bias caused by differential privacy noise, optimizing the system model's utility. Specifically, an efficient matrix factorization recommendation mechanism is proposed, where user data, after undergoing random responses locally, is uploaded via multiple messages and shuffled and statistically analyzed on the server side, achieving efficient estimation of the user feature matrix and item feature matrix, thereby constructing a privacy-preserving recommendation mechanism. Furthermore, this invention derives a privacy amplification limit for the multi-message protocol, further optimizing the utility of the recommendation mechanism. Under the same privacy budget, the error of this invention is reduced by 50%-80% compared to traditional local differential privacy methods.

[0107] Step S4: Aggregate and traverse the shuffled message dataset, then update the project feature matrix information on the server side and the user feature matrix information on the user side;

[0108] Preferably, in step S4, the aggregated and shuffled message dataset includes: a global message set shuffled by the server, and mean calculation of the perturbation gradient information for each data dimension. Since the client performs dimensionality reduction, the low-rank gradient matrix needs to be reconstructed on the server side. For each latitude point The statistics include the gradient set at that latitude, and the gradient set... ,in, This represents the obfuscated message dataset; Indicates user The generated message set; This represents the set union operation; express The perturbation gradient value at the location; Represents a set Index information in the elements of a triplet;

[0109] Next, the server calculates the mean of the messages for each data dimension, where the dimension point... The element value is , express The gradient value in; This represents the set of gradients.

[0110] This step optimizes the processing and updating of gradient information by reconstructing the low-rank gradient matrix on the server side and averaging the perturbation gradients for each data dimension. Specifically, by shuffling the global message set, statistically analyzing the gradient set for each data dimension, and averaging the results, it ensures that true gradient information can be effectively extracted when dealing with large amounts of perturbation data. This reduces redundancy and interference in the data, thereby improving the training efficiency and stability of the model, ensuring the reliability and accuracy of gradient information, enhancing the stability and precision of model training, reducing computational complexity and storage requirements, enhancing the model's generalization ability, and ensuring a more efficient and accurate training process.

[0111] Preferably, in step S4, updating the project feature matrix information on the server side includes: the server first uses a pseudo-inverse random dimensionality reduction matrix. Reconstruct the project gradient matrix ,in Then, the updated project feature matrix is ​​calculated using the reconstructed project gradient matrix. The calculation formula is as follows:

[0112]

[0113] in, Represents the project feature matrix; This refers to the learning rate in this update. This represents the penalty parameter for the weights of the project's feature vectors.

[0114] Preferably, in step S4, updating the user feature matrix information on the user end includes: to reduce communication overhead in this process, the user... Download the reconstructed low-rank gradient matrix The project gradient matrix is ​​reconstructed locally, and then the updated gradient of the user feature vector is calculated using the following formula:

[0115]

[0116] in, Represents user feature vector The update gradient; Indicates a marker variable; Represents the feature vector of the project; Represents each user Having the project Preference rating; Represents user feature vector Transpose of; Indicates the number of recommended items;

[0117] And by using this gradient information to update the user feature vector, we can obtain...

[0118]

[0119] in, This represents the updated user feature vector; This refers to the learning rate in this update. This represents the penalty parameter for the weights of the user feature vector.

[0120] This step reconstructs the project gradient matrix on the server side using a pseudo-inverse random dimensionality reduction matrix, projects the feature gradient matrix into a low-rank space, and updates the project feature matrix using the reconstructed matrix. This improves the efficiency and accuracy of updating the project feature matrix, allowing users to complete the feature vector update operation simply by downloading a low-rank update matrix with fewer parameters, further reducing the communication overhead between the user and the server. Simultaneously, the user client optimizes the calculation and update process of the user feature matrix by downloading the reconstructed low-rank gradient matrix and performing local updates, effectively reducing data transmission volume, lowering computational and communication overhead on the user side, and maintaining good computational stability and convergence in a distributed environment. This improves the model's generalization ability, avoids overfitting, reduces computational overhead, and enhances the overall system performance.

[0121] Step S5: Perform privacy budget segmentation to ensure that differential privacy constraints are met in each iteration of the model. Finally, integrate the user feature matrix and the item feature matrix to construct a matrix factorization recommendation model.

[0122] Preferably, in step S5, when performing privacy budget segmentation, the matrix factorization algorithm needs to achieve model convergence in multiple iterations. Let the number of iterations required by the model be... Based on the fundamental differential privacy combination mechanism, the privacy budget for each round needs to be set as follows: To ensure that the final algorithm meets the requirements. -DP constraint.

[0123] In the matrix factorization recommendation model constructed by integrating the user feature matrix and the item feature matrix, the calculation of the user feature matrix is ​​performed on the local device to protect user privacy. To allow the server to independently complete the prediction process, each user needs to submit their local feature vector after the algorithm iteration is complete. Uploaded to the server. Because the entire algorithm mechanism follows differential privacy constraints, this step does not pose a threat to user privacy.

[0124] Preferably, such as Figure 2As shown, in this embodiment, to further verify the effectiveness of the invention, experiments were also conducted on two real-world datasets (MovieLens 1M and 10M) to compare the model prediction error rates under different privacy budgets. The MovieLens dataset includes user ratings of their preferences for different movies, with the rating values ​​located at... Range. Regarding parameter settings, the penalty parameter... and Set all to Specifically, in the MovieLens 1M dataset, the latent vector data dimension is set to... The dimension of the low-rank space vector is Learning rate In the MovieLens10M dataset, the latent vector data dimension is set to... The dimension of the low-rank space vector is Learning rate The set of privacy budget parameters for testing is as follows: The root mean square error (RMSE) is used to evaluate model performance, and its formula is as follows:

[0125]

[0126] in, For the true result, The model's prediction results are shown in Figure 2. The experimental results demonstrate that, compared to traditional local difference privacy matrix factorization mechanisms, the method presented in this embodiment reduces the classification error rate by 15%-50% while maintaining privacy security, and significantly reduces the privacy budget. More specifically:

[0127] In a strong privacy protection environment (privacy budget) On the MovieLens 1M dataset, the traditional local differential privacy method achieves an RMSE of 0.51, while this embodiment achieves an RMSE of 0.43, representing a 15% improvement. When processing a larger dataset (MovieLens 10M), the traditional local differential privacy method achieves an RMSE of 0.21, while this embodiment achieves an RMSE of 0.11, representing an approximately 16% improvement.

[0128] In environments with weak privacy protection (privacy budget) On the MovieLens 1M dataset, the RMSE of the traditional local differential privacy method is 0.455, while the RMSE of this embodiment is 0.425, showing a significant improvement. When dealing with a larger dataset (MovieLens 10M), the RMSE of the traditional local differential privacy method is 0.21, while the RMSE of this invention is 0.11, representing an improvement of approximately 50%.

[0129] Meanwhile, it can be observed that when the amount of data used to train the model is relatively large, the method provided in this embodiment has a stable improvement over the traditional local differential privacy method, that is, there are visible differences under different budgets.

[0130] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A matrix factorization recommendation method based on shuffle differential privacy, the method comprising: acquiring an interaction dataset between users and items, and inputting it into a matrix factorization recommendation model for processing, thereby obtaining item recommendation results for users; Its features are, It also includes training the matrix factorization recommendation model, the training steps of which are as follows: S1: Preprocess model information on the server side, including setting system parameters, initializing system information, and setting optimization goals; S2: Calculate the user feature matrix on the user end based on the preprocessed model information, and generate a message set by locally perturbing each message. Then, upload the generated message set to the shuffling server. S3: The shuffling server receives messages from the user end after local perturbation, and uses a differential privacy shuffling model to shuffle these messages to obtain a shuffled message dataset, which is then transmitted to the server. S4: Aggregate and traverse the shuffled message dataset, then update the project feature matrix information on the server side and the user feature matrix information on the user side; S5: Perform privacy budget segmentation to ensure that the differential privacy constraint is met in each iteration of the model. Finally, integrate the user feature matrix and the item feature matrix to construct the matrix factorization recommendation model. Step S2 includes: S21: Each user uses their local preference rating to calculate the recommendation feature matrix and update the gradient; S22: Perform random sampling to constrain the gradient values ​​within the desired data domain; S23: Locally scramble the sampled data points; S24: Generate a message set by splitting the perturbation result into independent messages and then adding them to the message set. If the size of the message set is less than the number of messages K generated by each user, repeat steps S22 and S23 until the size of the message set is equal to the number of messages K generated by each user. S25: Upload the generated message set to the shuffling server; In step S4, the aggregated and shuffled message dataset includes: a global message set shuffled by the server, mean calculation of the perturbation gradient information for each data dimension, and reconstruction of the low-rank gradient matrix on the server side. For each latitude point The statistics include the gradient set at that latitude, and the gradient set... ,in, This represents the obfuscated message dataset; Indicates user The generated message set; This represents the set union operation; express The perturbation gradient value at the location; Represents a set Index information in the elements of a triplet; Next, the server calculates the mean of the messages for each data dimension, where the dimension point... The element value is , express The gradient value in; This represents the set of gradients.

2. The matrix factorization recommendation method based on shuffle differential privacy according to claim 1, characterized in that, Step S21 includes: S211: For each user Existing projects Set a marker variable for the rating record. The value is 1 if it is not 1, otherwise it is 0. S212: Calculate the gradient vector using local rating record data, and calculate the vector in the low-rank space using a randomized dimensionality reduction matrix.

3. The matrix factorization recommendation method based on shuffle differential privacy according to claim 2, characterized in that, In step S212, the gradient vector calculation formula is: in, Represents the gradient vector; Indicates a marker variable; Represents the user feature vector; Represents each user Having the project Preference rating; express Transpose of; Represents the feature vector of the project; This represents the number of recommended items; the formula for calculating the vector in the low-rank space is: in, Represents a low-rank space vector; This represents a randomized reduced-dimensional matrix.

4. The matrix factorization recommendation method based on shuffle differential privacy according to claim 1, characterized in that, Step S22 includes: S221: The user samples two dimensional variables from a uniform distribution. and ,in, The sampling set is , The sampling set is ,in Represents the dimension of a low-rank spatial vector. The dimensions representing user features and product feature vectors; S222: Let and The low-rank gradient values ​​corresponding to the two latitudes are ,like The value is outside the data field If so, the data needs to be pruned to constrain the gradient value within the desired data domain.

5. The matrix factorization recommendation method based on shuffle differential privacy according to claim 4, characterized in that, In step S23, the local perturbation of the sampled data points includes: In a Bernoulli distribution, the variable is identified from the sampling sign. ,if A value of 1 returns the perturbation gradient. Otherwise return ,in Represents the dimension of the implicit vector data; Privacy parameters indicating user disruption mechanisms; This represents the total number of iterations of the algorithm.

6. The matrix factorization recommendation method based on shuffle differential privacy according to claim 1, characterized in that, Step S3 includes: S31: Summarize the noise data in all user-submitted disturbance messages and integrate them into a temporary dataset according to a unified format, eliminating user labeling information; S32: Perform a reverse traversal random permutation on the temporary dataset. Starting from the last data, randomly select a data from a previous position each time and swap it with the current position until the traversal is complete, ensuring that all data positions are uniformly and randomly shuffled. S33: Transmit the obfuscated message dataset to the server.

7. The matrix factorization recommendation method based on shuffle differential privacy according to claim 1, characterized in that, In step S4, updating the project feature matrix information on the server side includes: the server first uses a pseudo-inverse random dimensionality reduction matrix. Reconstruct the project gradient matrix ,in Then, the updated project feature matrix is ​​calculated using the reconstructed project gradient matrix. The calculation formula is as follows: in, Represents the project feature matrix; This refers to the learning rate in this update. This represents the penalty parameter for the weights of the project's feature vectors.

8. The matrix factorization recommendation method based on shuffle differential privacy according to claim 1, characterized in that, In step S4, updating the user feature matrix information on the user end includes: user Download the reconstructed low-rank gradient matrix The project gradient matrix is ​​reconstructed locally, and then the updated gradient of the user feature vector is calculated using the following formula: in, Represents user feature vector The update gradient; Indicates a marker variable; Represents the feature vector of the project; Represents each user Having the project Preference rating; Represents user feature vector Transpose of; This represents the number of recommended items; and by using this gradient information to update the user feature vector, we can obtain... in, This represents the updated user feature vector; This refers to the learning rate in this update. This represents the penalty parameter for the weights of the user feature vector.

Images