Project recommendation method for multi-view federated matrix decomposition based on differential privacy protection

Abstract

The invention discloses a project recommendation method for multi-view federated matrix decomposition based on differential privacy protection, which comprises the following steps that: 1, a federated server initializes a user feature implicit factor matrix and a project implicit factor matrix, and distributes the user feature implicit factor matrix and the project implicit factor matrix to a user side and a project server; 2, a user side updates a user implicit factor vector and an average score vector based on local data, and uploads a cutting gradient through adaptive gradient cutting and differential privacy noise injection; 3, the project server updates a project feature implicit factor matrix by using the project feature data, and uploads a gradient to a federated server; 4, the federal server aggregates gradients of all directions, updates the implicit factor matrix and performs iterative training; and 5, generating a final recommendation model after training is completed. Multi-view data such as a scoring matrix, a user feature matrix and a project feature matrix are fused, and meanwhile, privacy protection and model performance are dynamically balanced by adopting self-adaptive gradient cutting.

Description

Technical Field

[0001] The present invention relates to a privacy-preserving recommendation system, and in particular to an item recommendation method of multi-view federated matrix decomposition with differential privacy preservation. Background Art

[0002] In the era of the Internet and big data, recommendation systems, as an important technical means, are widely used in e-commerce, social media, content distribution and other fields. Recommendation systems analyze users' historical behavior data, explore their interests and preferences, and recommend products, content or services that may be of interest to users. As data privacy issues become increasingly prominent, protecting user privacy has become an important consideration in the design of recommendation systems.

[0003] Differential privacy is a mathematical framework designed to provide strong privacy protection. By adding noise to the data processing process, differential privacy ensures that the presence or absence of any single data point does not significantly affect the output results, thereby preventing personal information leakage. In the recommendation system, differential privacy can be used to protect the user's behavior data so that even if the attacker obtains the data, the details of the specific user cannot be inferred. Federated learning is a distributed machine learning method that allows the model to be trained on multiple local devices (such as the user's mobile phone or computer) without uploading the data to a central server. Each local device only needs to send model updates (such as gradients) to the central server, which aggregates these updates and updates the global model, and then sends the updated model back to the local device. This approach not only reduces the bandwidth requirements for data transmission, but also effectively protects the privacy of users.

[0004] Purpose of the Invention

[0005] The present invention aims to address the deficiencies of the above-mentioned prior art and proposes a project recommendation method of multi-view federated matrix decomposition with differential privacy protection, in order to achieve efficient personalized project recommendation while meeting the privacy protection requirements, and overcome the problems existing in the prior art such as high risk of privacy leakage, low data collaboration efficiency and limited project recommendation performance.

[0006] The technical solution adopted by the present invention to solve the technical problem is:

[0007] The project recommendation method of the differential privacy-preserving multi-view federated matrix decomposition of the present invention is characterized in that it is applied to a federated server, a project server, In a network consisting of 100 clients, each client has the same Project, The rating dataset owned by a user is denoted as ,in, Indicates The rating of the jth item by the user. If there is no rating for the jth item, let ; said The user feature dataset owned by a client is recorded as ,in, Indicates The first user User characteristics, is the total number of user features; the project feature data set owned by the project server is recorded as ,in, Indicates Project No. Project features, is the total number of project features; the project recommendation method is performed in the following steps:

[0008] Step 1: The federated server randomly initializes the user feature latent factor matrix , the item latent factor matrix ,in, is the dimension of latent factors; let The Row vector Indicates User characteristics The latent factor vector of Middle Row vector Indicates latent factor vector of items;

[0009] Step 2: The federated server converts the user feature latent factor matrix and the item latent factor matrix Distribute to user terminals and the item latent factor matrix Sending to the project server;

[0010] Step 3: Define the current round number as The total number of rounds is , and initialize ;

[0011] Step 4: The user terminal randomly initializes the User latent factor vector under round And calculate the user average rating vector ; and according to the user feature latent factor matrix and the item latent factor matrix right and Update to calculate the The gradient of the latent factor matrix of the project after the Gaussian noise perturbation and Gradient of user feature latent factor matrix after round of Gaussian noise perturbation and sent to the federated server;

[0012] Step 4.1: The user terminal randomly initializes the User latent factor vector under round And calculate the user average rating vector ; said Client No. User feature latent factor matrix under round = ; said Client No. The latent factor matrix of the project under the round = ;

[0013] Step 4.2: Clients pass and calculate Gradient , and the gradient After clipping, the first The clipping threshold under the wheel ;

[0014] Step 4.3: Client Utilization Calculate the item latent factor matrix Gradient , the gradient After cropping, Gaussian noise satisfying differential privacy is injected to calculate the Wheel clipping threshold ;

[0015] Step 4.4: Client pair and Update to get User latent factor vector under round and the user average rating vector ;

[0016] Step 5: The project server randomly initializes The latent factor matrix of project features under the round , and based on the project feature dataset ,use right Update to calculate the Round Item Factor Matrix Gradient and sent to the federated server;

[0017] Step 6: The federated server receives and aggregates the perturbed gradients sent by the client and the gradients sent by the project server, and and Update to get The user feature latent factor matrix of round and the item latent factor matrix , and and Distribute to A user terminal will Sending to the project server;

[0018] Step 7: Judgement Is it established? If established, Assign to After that, return to step 4.2 to execute sequentially; otherwise, , and As the optimal user latent factor vector , optimal user average rating vector and the optimal item latent factor matrix ;

[0019] Step 8: Use formula (20) to calculate the The user end Predicted ratings for items ; thus obtaining the The predicted rating set of all items by the user ;

[0020] (20)

[0021] Step 9: From Select the top N items with the highest predicted scores as the recommendation results and send them to the A user terminal.

[0022] The project recommendation method of the differential privacy-preserving multi-view federated matrix decomposition according to the present invention is also characterized in that the step 4.2 comprises:

[0023] Step 4.2.1: The user terminal uses formula (1) to calculate the Gradient under the wheel :

[0024] (1)

[0025] Step 4.2.2: The user end uses formula (2) to obtain Gradient after clipping under the wheel ;

[0026] (2)

[0027] In formula (2), Indicates The clipping threshold of the gradient of the round user feature latent factor matrix; express -norm;

[0028] Step 4.2.3: The user end uses formula (3) to obtain Gradient of user feature latent factor matrix after round of Gaussian noise perturbation ;

[0029] (3)

[0030] In formula (3), Indicates The user characteristic latent factor noise scale of the round, and , is the failure probability, Budget for privacy; represents Gaussian distribution;

[0031] Step 4.2.4: Use equation (4) to calculate the Wheel clipping threshold :

[0032] (4)

[0033] In formula (4), are predefined parameters; express of -norm.

[0034] Further, the step 4.3 includes:

[0035] Step 4.3.1: The user terminal uses formula (5) to calculate the Gradient under the wheel ;

[0036] (5)

[0037] In formula (5), Indicates The latent factor matrix of the round item No. row vector;

[0038] Step 4.3.2: The user terminal uses formula (6) to obtain Gradient after round clipping ;

[0039] (6)

[0040] In formula (6), Indicates The clipping threshold of the gradient of the round-item latent factor matrix;

[0041] Step 4.3.3: The user end uses formula (7) to obtain The gradient of the latent factor matrix of the project after the Gaussian noise perturbation ;

[0042] (7)

[0043] In formula (7), Indicates The latent factor noise scale of the round, and ;

[0044] Step 4.3.4: Use equation (8) to calculate The pruning threshold of the round ;

[0045] (8).

[0046] Furthermore, in step 4.4, equation (9)-equation (10) is used to obtain User latent factor vector under round and the user average rating vector ;

[0047] (9)

[0048] (10)

[0049] In formula (9)-formula (10), is the learning rate, Indicates User latent factor vector under round The gradient of Indicates User rating vector for the round The gradient of , and:

[0050] (11)

[0051] (12)

[0052] In formula (11), T represents transposition.

[0053] Further, the step 5 comprises:

[0054] Step 5.1: The project server uses equation (13)-equation (14) to Middle Line item feature latent factor vector Update to get Round Line item feature latent factor vector ;

[0055] (13)

[0056] (14)

[0057] In formula (13), express The gradient of

[0058] Step 5.2: The project server uses formula (15) to calculate the Round item latent factor matrix Gradient :

[0059] (15).

[0060] Furthermore, in step 6, equations (16) and (17) are used to respectively calculate The user feature latent factor matrix of round Middle Row vector and the item latent factor matrix No. Row vector Update to get The user feature latent factor matrix of round Middle Row vector and the item latent factor matrix No. Row vector , and and Distribute to A user terminal will Sending to the project server;

[0061] (16)

[0062] (17)

[0063] In formula (16) and formula (17), express The gradient of express The gradient of , and:

[0064] (18)

[0065] (19)

[0066] In formula (17) and formula (18), To control the parameters, is the regularization parameter.

[0067] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the item recommendation method, and the processor is configured to execute the program stored in the memory.

[0068] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the item recommendation method when executed by a processor.

[0069] Compared with the solutions in the prior art, the beneficial effects of the present invention are as follows:

[0070] 1. This invention effectively protects the privacy of users in the data sharing process by combining differential privacy and federated learning. The original data of users is retained locally, and only the gradient information perturbed by noise is transmitted between users and servers, thereby reducing the risk of sensitive information leakage. By dynamically adjusting the gradient clipping threshold and optimizing model parameters, the convergence of the model and the strength of privacy protection are balanced, further improving the efficiency of project recommendation.

[0071] 2. The present invention utilizes multi-view data (such as rating matrix, user feature matrix and project feature matrix) to perform matrix decomposition and fuse multi-dimensional feature information, thereby significantly improving the accuracy of project recommendations.

[0072] 3. This invention adopts a federated learning framework, where most of the calculations are performed on the user side, and the server side is only responsible for aggregating the perturbed gradients and updating the model parameters. This distributed computing method effectively reduces the computing pressure and data storage requirements on the server side, while reducing communication overhead and improving the efficiency and scalability of project recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is a schematic diagram of multi-view matrix decomposition;

[0074] Figure 2 It is a structural diagram of the system of the present invention. DETAILED DESCRIPTION

[0075] In this embodiment, a project recommendation method for multi-view federated matrix decomposition with differential privacy protection is applied to a federated server, a project server, In a network consisting of 100 clients, each client has the same Projects, User-side The rating data set we have is .in, Indicates The user's rating of the j-th item, if there is no rating for the j-th item, then . Client The user feature data set is ,in, Indicates The first user eigenvalues, is the total number of user features. The project feature data set owned by the project server is ,in, Indicates Project No. eigenvalues, is the total number of project features.

[0076] like Figure 1 As shown, multi-view matrix decomposition is performed on three data sources: (1) the rating matrix , (2) User feature matrix , (3) Project feature matrix .for Has Users with characteristics Has For items with features, the decomposition of these three matrices is as follows:

[0077] (1)

[0078] The global objective function is defined as formula (2):

[0079] (2)

[0080] In formula (2), For the The weight of the error term of the jth item by the user end is , then let ,like , then let , is the uncertainty coefficient of the unrated items, Reduces the amount of information exposed to the federation server about which items users have rated, and , To control the parameters, Controls how strongly the model shares information from side view data, , Restrict the model to not learn any side view data. is the regularization parameter. In this example, , T represents transpose, such as Figure 2 As shown, the project recommendation method includes the following steps:

[0081] Step 1: The federated server randomly initializes the user feature latent factor matrix , the item latent factor matrix ,in, is the dimension of the latent factor, in this example The value of is 20; The Row vector Indicates User characteristics The latent factor vector of Middle Row vector Indicates latent factor vector of items;

[0082] Step 2: The federated server converts the user feature latent factor matrix and the item latent factor matrix Distribute to user terminals and the item latent factor matrix Sent to the project server.

[0083] Step 3: Define the current round number as The total number of rounds is , in this example The value of is 30, which means the number of rounds of federated training is 30, and the initialization ;

[0084] Step 4: The user terminal randomly initializes the User latent factor vector under round And calculate the user average rating vector ; and according to the user feature latent factor matrix and the item latent factor matrix right and Update to calculate the gradient and And send it to the federated server.

[0085] Step 4.1: The user terminal randomly initializes the User latent factor vector under round And calculate the user average rating vector ;No. Client No. User feature latent factor matrix under round = ;No. Client No. The latent factor matrix of the project under the round = ;

[0086] Step 4.2: Clients pass and calculate Gradient , and the gradient After clipping, the first The clipping threshold under the wheel .

[0087] Step 4.2.1: The user terminal uses formula (3) to calculate the Gradient under the wheel :

[0088] (3)

[0089] Step 4.2.2: The user end uses formula (4) to obtain Gradient after clipping under the wheel ;

[0090] (4)

[0091] In formula (4), Indicates The clipping threshold of the gradient of the round user feature latent factor matrix; express -norm;

[0092] Step 4.2.3: The user end uses formula (5) to obtain Gradient of user feature latent factor matrix after round of Gaussian noise perturbation ;

[0093] (5)

[0094] In formula (5), Indicates The user characteristic latent factor noise scale of the round, and , is the failure probability, Budget for privacy, The smaller the value, the higher the privacy protection. The gradient is perturbed using the Gaussian mechanism to prevent reconstruction attacks on the gradient. Adjust according to the specific privacy protection requirements. In this example, , ; Represents a Gaussian distribution.

[0095] Step 4.2.4: Use equation (6) to calculate the Wheel clipping threshold :

[0096] (6)

[0097] In formula (6), is the attenuation factor, which adjusts the size of each round of clipping threshold. In this example, ; express of -norm.

[0098] Step 4.3: Client Utilization Calculate the item latent factor matrix Gradient , the gradient After cropping, Gaussian noise satisfying differential privacy is injected to calculate the Wheel clipping threshold ;

[0099] Step 4.3.1: The user terminal uses formula (7) to calculate the Gradient under the wheel ;

[0100] (7)

[0101] In formula (7), Indicates The latent factor matrix of the round item No. Row vector.

[0102] Step 4.3.2: The user terminal uses formula (8) to obtain Gradient after clipping ;

[0103] (8)

[0104] In formula (8), Indicates The clipping threshold of the gradient of the round-item latent factor matrix;

[0105] Step 4.3.3: The user end uses formula (9) to obtain The gradient of the latent factor matrix of the project after the Gaussian noise perturbation ;

[0106] (9)

[0107] In formula (9), Indicates The latent factor noise scale of the round, and .

[0108] Step 4.3.4: Use equation (10) to calculate The pruning threshold of the round ;

[0109] (10)

[0110] Step 4.4: Each user terminal uses formula (11)-formula (12) to and Update to get User latent factor vector under round and the user average rating vector ;

[0111] (11)

[0112] (12)

[0113] In formula (11)-formula (12), is the learning rate. In this example, the learning rate ; Indicates User latent factor vector under round The gradient of Indicates User rating vector for the round The gradient of , and:

[0114] (13)

[0115] (14)

[0116] Step 4.5: The client will and Sent to the federation server.

[0117] Step 5: The project server randomly initializes The latent factor matrix of project features under the round , and based on the project feature dataset ,use right Update to calculate the Round Item Factor Matrix Gradient And send it to the federated server.

[0118] Step 5.1: The project server uses equation (15)-equation (16) to Middle Line item feature latent factor vector Update to get Round Line item feature latent factor vector ;

[0119] (15)

[0120] (16)

[0121] In formula (15), express gradient.

[0122] Step 5.2: The project server uses equation (17) to calculate the Round item latent factor matrix Gradient and sent to the federated server;

[0123] (17).

[0124] Step 6: The federated server receives and aggregates the perturbed gradients sent by the client and the gradients sent by the project server, and then uses equations (18) and (19) to calculate the perturbed gradients of the first The user feature latent factor matrix of round Middle Row vector and the item latent factor matrix No. Row vector Update to get The user feature latent factor matrix of round Middle Row vector and the item latent factor matrix No. Row vector , and and Distribute to A user terminal will Send to the project server;

[0125] (18)

[0126] (19)

[0127] In formula (18) and formula (19), express The gradient of express The gradient of , and:

[0128] (20)

[0129] (twenty one).

[0130] Step 7: Judgement Is it established? If established, Assign to After that, return to step 4.2 to execute sequentially; otherwise, , and As the optimal user latent factor vector , optimal user average rating vector and the optimal item latent factor matrix ;

[0131] Step 8: Use formula (22) to calculate the The user end Predicted ratings for items ; thus obtaining the The predicted rating set of all items by the user ;

[0132] (twenty two).

[0133] Step 9: From Select the top N items with the highest predicted scores as the recommendation results and send them to the A user terminal.

[0134] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0135] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium, and the computer program executes the steps of the above method when executed by a processor.

[0136] In summary, the method of the present invention effectively solves the balance problem between protecting user privacy and improving recommendation quality in the recommendation system. First, the present invention integrates multi-view data, including rating matrix, user feature matrix and project feature matrix, and significantly improves the accuracy of the recommendation system through joint matrix decomposition. Secondly, by adopting the federated learning framework, each participant can realize distributed collaborative training without sharing the original data, which effectively protects the user's sensitive information and solves the risk of data privacy leakage. Third, the present invention introduces an adaptive gradient clipping strategy, dynamically adjusts the clipping threshold, and combines the differential privacy noise injection mechanism to improve the performance and convergence speed of the model while ensuring privacy protection.

Claims

1. A project recommendation method based on multi-view federated matrix decomposition with differential privacy preservation, characterized in that: It is applied to the federated servers, project servers, In a network consisting of 100 clients, each client has the same Project, The rating dataset owned by a user is denoted as ,in, Indicates The rating of the jth item by the user. If there is no rating for the jth item, let ; said The user feature dataset owned by a client is recorded as ,in, Indicates The first user User characteristics, is the total number of user features; the project feature data set owned by the project server is recorded as ,in, Indicates Project No. Project features, is the total number of project features; the project recommendation method is performed in the following steps: Step 1: The federated server randomly initializes the user feature latent factor matrix , the item latent factor matrix ,in, is the dimension of latent factors; let The Row vector Indicates User characteristics The latent factor vector of Middle Row vector Indicates latent factor vector of items; Step 2: The federated server converts the user feature latent factor matrix and the item latent factor matrix Distribute to user terminals and the item latent factor matrix Sending to the project server; Step 3: Define the current round number as The total number of rounds is , and initialize ; Step 4: The user terminal randomly initializes the User latent factor vector under round And calculate the user average rating vector ; and according to the user feature latent factor matrix and the item latent factor matrix right and Update to calculate the The gradient of the latent factor matrix of the project after the Gaussian noise perturbation and Gradient of user feature latent factor matrix after round of Gaussian noise perturbation and sent to the federated server; Step 4.1: The user terminal randomly initializes the User latent factor vector under round And calculate the user average rating vector ; said Client No. User feature latent factor matrix under round = ; said Client No. The latent factor matrix of the project under the round = ; Step 4.2: Clients pass and calculate Gradient , and the gradient After clipping, the first The clipping threshold under the wheel ; Step 4.3: Client Utilization Calculate the item latent factor matrix Gradient , the gradient After cropping, Gaussian noise satisfying differential privacy is injected to calculate the Wheel clipping threshold ; Step 4.4: Client pair and Update to get User latent factor vector under round and the user average rating vector ; Step 5: The project server randomly initializes The latent factor matrix of project features under the round , and based on the project feature dataset ,use right Update to calculate the Round Item Factor Matrix Gradient and sent to the federated server; Step 6: The federated server receives and aggregates the perturbed gradients sent by the client and the gradients sent by the project server, and and Update to get The user feature latent factor matrix of round and the item latent factor matrix , and and Distribute to A user terminal will Sending to the project server; Step 7: Judgement Is it established? If established, Assign to After that, return to step 4.2 to execute sequentially; otherwise, , and As the optimal user latent factor vector , optimal user average rating vector and the optimal item latent factor matrix ; Step 8: Use formula (20) to calculate the The user end Predicted ratings for items ; thus obtaining the The predicted rating set of all items by the user ; (20) Step 9: From Select the top N items with the highest predicted scores as the recommendation results and send them to the A user terminal.

2. The project recommendation method of multi-view federated matrix decomposition with differential privacy protection according to claim 1, characterized in that: The step 4.2 comprises: Step 4.2.1: The user terminal uses formula (1) to calculate the Gradient under the wheel : (1) Step 4.2.2: The user end uses formula (2) to obtain Gradient after clipping under the wheel ; (2) In formula (2), Indicates The clipping threshold of the gradient of the round user feature latent factor matrix; express -norm; Step 4.2.3: The user end uses formula (3) to obtain Gradient of user feature latent factor matrix after round of Gaussian noise perturbation ; (3) In formula (3), Indicates The user characteristic latent factor noise scale of the round, and , is the failure probability, Budget for privacy; represents Gaussian distribution; Step 4.2.4: Use formula (4) to calculate the Wheel clipping threshold : (4) In formula (4), are predefined parameters; express of -norm.

3. The project recommendation method of multi-view federated matrix decomposition with differential privacy protection according to claim 2, characterized in that: The step 4.3 comprises: Step 4.3.1: The user terminal uses formula (5) to calculate the Gradient under the wheel ; (5) In formula (5), Indicates The latent factor matrix of the round item No. row vector; Step 4.3.2: The user terminal uses formula (6) to obtain Gradient after clipping ; (6) In formula (6), Indicates The clipping threshold of the gradient of the round-item latent factor matrix; Step 4.3.3: The user end uses formula (7) to obtain The gradient of the latent factor matrix of the project after the Gaussian noise perturbation ; (7) In formula (7), Indicates The latent factor noise scale of the round, and ; Step 4.3.4: Use equation (8) to calculate The pruning threshold of the round ; (8)。 4. The project recommendation method of multi-view federated matrix decomposition with differential privacy protection according to claim 3, characterized in that: In step 4.4, equation (9)-equation (10) is used to obtain User latent factor vector under round and the user average rating vector ; (9) (10) In formula (9)-formula (10), is the learning rate, Indicates User latent factor vector under round The gradient of Indicates User rating vector for the round The gradient of , and: (11) (12) In formula (11), T represents transposition.

5. The project recommendation method of multi-view federated matrix decomposition with differential privacy protection according to claim 4, characterized in that: The step 5 comprises: Step 5.1: The project server uses equation (13)-equation (14) to Middle Line item feature latent factor vector Update to get Round Line item feature latent factor vector ; (13) (14) In formula (13), express The gradient of Step 5.2: The project server uses formula (15) to calculate the Round item latent factor matrix Gradient : (15)。 6. The project recommendation method of multi-view federated matrix decomposition with differential privacy protection according to claim 5, characterized in that: In step 6, equations (16) and (17) are used to respectively calculate The user feature latent factor matrix of round Middle Row vector and the item latent factor matrix No. Row vector Update to get The user feature latent factor matrix of round Middle Row vector and the item latent factor matrix No. Row vector , and and Distribute to A user terminal will Sending to the project server; (16) (17) In formula (16) and formula (17), express The gradient of express The gradient of , and: (18) (19) In formula (17) and formula (18), To control the parameters, is the regularization parameter.

7. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the item recommendation method described in any one of claims 1 to 6, and the processor is configured to execute the program stored in the memory.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the item recommendation method according to any one of claims 1 to 6 are executed.

Images