Matrix decomposition method based on secure aggregation and key exchange

Abstract

The application discloses a matrix decomposition method based on secure aggregation and key exchange, and provides a new idea for enhancing data security of federated learning by performing secure aggregation on the gradient of an item matrix I of matrix decomposition under a federated learning framework; the training sample of a recommendation model (namely, a federated learning model) is efficiently utilized by using the local and securely aggregated gradient, so that the user data is ensured not to leave the local, and meanwhile, the recommendation model training process is made more secure; the gradient is masked and added with noise, so that the leakage of source data information caused by exposure of the real gradient is effectively avoided; and the gradient aggregation mode based on secure aggregation is provided, and compared with the homomorphic encryption technology adopted in the background art, the gradient encryption and decryption have lower calculation complexity and faster calculation speed, and the training speed of the recommendation model is improved.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and more specifically to a matrix decomposition method based on secure aggregation and key exchange. Background Technology

[0002] Currently, secure matrix factorization algorithms are mainly based on distributed matrix factorization algorithms and employ encryption techniques such as Paillier homomorphic encryption to ensure the security of transmitted information and prevent the leakage of user's local data. The main implementation steps of existing secure matrix factorization algorithms are as follows:

[0003] 1. The server initializes the item matrix I, and the client initializes its own user matrix U locally. The public key is shared by both the server and the client, while the private key is only available to the client.

[0004] 2. The server uses the public key to encrypt I to obtain ciphertext C. I Then broadcast to all clients;

[0005] 3. Each client receives C I Then use the local private key to access C I Decryption yields the actual item matrix I, which is used to calculate the gradient of the client's held item U and update U accordingly. After updating, the gradient G of I is calculated and encrypted to obtain the ciphertext C. G ;

[0006] 4. Server-side collection of C G And update to get C I =C I -C G Then update C I Broadcast to all clients;

[0007] 5. Repeat steps 3-4 until the algorithm converges.

[0008] As seen in steps 1-5 above, the existing solution ensures that user data does not leave the local machine, and homomorphic encryption prevents the server from obtaining the plaintext gradients throughout the entire training process, thus making it impossible to deduce the original data from a single gradient. However, the homomorphic encryption solution requires repeated encryption and decryption, making training inefficient. If homomorphic encryption is removed and the plaintext gradients of a single data point are directly summarized, the original data can be deduced after multiple training steps, but the security of local data cannot be guaranteed. Therefore, how to solve the above-mentioned technical problems of existing secure matrix factorization algorithms has become an urgent problem to be solved in the industry. Summary of the Invention

[0009] This invention aims to make the recommendation model training process more efficient and ensure that local data is not leaked during model training. It provides a matrix factorization method based on secure aggregation and key exchange.

[0010] To achieve this objective, the present invention adopts the following technical solution:

[0011] A matrix factorization method based on secure aggregation and key exchange is provided, comprising the following steps:

[0012] S1, let the scheduler of the federated learning framework be the server and each participant in the training be the client. The server will broadcast the initialized item embedding matrix I to each client.

[0013] S2, each client X uses the embedding matrix I to calculate the embedding matrix U for its respective local user. X gradient and utilize Update the local user's embedded matrix U X ;

[0014] S3, each client X utilizes the locally updated U X Calculate the gradient generated with respect to the embedding matrix I.

[0015] S4, Update the gradient using a key exchange method. And on Summarize to obtain Then, using Update the embedding matrix I;

[0016] S5. Repeat steps S2-S4 until the termination condition of federated learning is met.

[0017] Preferably, in step S2, the embedding matrix The embedding vector of the local user i gradient It is calculated using the following formula (1):

[0018]

[0019] In formula (1), L is the loss function for client X in federated learning.

[0020] M X This represents the rating matrix at client X;

[0021] I T It is the matrix transpose of I;

[0022] ||·|| F Denotes the Frobenius norm of a matrix;

[0023] I j ∈R 1×kThe embedding vector of item j shared by all clients is an embedding matrix I = [I1, I2, ..., I...]. j , ..., I d ]∈R d×k The j-th row;

[0024] Indicate I j The vector transpose;

[0025] This represents the rating of user i about item j held by client X (missing items where user i has no actual rating for item j are to be predicted after modeling).

[0026] This indicates that the client X owns the item j that user i has actually rated;

[0027] This represents the summation of the items j that user i has actually rated with respect to the symbol j, which is owned by client X.

[0028] Preferably, in step S2, the local user embedding matrix of each client X is updated using the following formula (2):

[0029]

[0030] In formula (2), λ U U X The regularization parameter,

[0031] Preferably, in step S3, the embedding vector I of the associated item j in the embedding matrix I is... j corresponding gradient It is calculated using the following formula (3):

[0032]

[0033] In formula (3), express The j-th row;

[0034] The embedding vector I represents the item j shared by all clients. j The vector transpose;

[0035] The embedding matrix U represents X The embedding vector associated with local user i;

[0036] This represents the rating of user i for item j that is locally owned by client X;

[0037] This refers to the users i owned by client X who have rated item j;

[0038] This represents the summation of the ratings of users i who have rated item j on client X, with respect to the symbol i.

[0039] Preferably, in step S4, the gradient is replaced. The key exchange method used is specifically as follows:

[0040] S41, Each client X generates a private key s locally. X and public key p X The server exchanges public keys generated by each client X, and each client X obtains a corresponding set of exchanged public keys, denoted as C. X ;

[0041] S42, according to C X and the private key s generated locally by each of the aforementioned clients X X A key agreement, denoted as key_agreement(X,Y), is generated between each client X and every other client Y.

[0042] S43, the client X uses the locally generated key_agreement(X, Y) as a seed to generate a mask, denoted as mask(X, Y), and then updates the gradient in step S3.

[0043] Preferably, in step S41, C X This is expressed by the following expression (4):

[0044] C X ={p1, ..., p X , ..., p N Expression (4)

[0045] In expression (4), This refers to the public key generated locally by client X;

[0046] p represents a prime number, which is agreed upon in advance by each client;

[0047] g represents the primitive root modulo p, which is agreed upon in advance by each client;

[0048] %p represents the modulo operation on a prime number p;

[0049] {p1,…,p X, ..., p N} represents the set of all N public keys generated locally by the clients and received by the server.

[0050] Preferably, in step S42, the method for generating key_agreement(X, Y) is as follows:

[0051] The client X obtains the public key exchange set C. X Extract the public key p of client Y from the data. Y ;

[0052] The client X uses the public key p Y and the locally generated private key s X Generate key_agreement(X, Y).

[0053] As a preferred option, the formula for generating key_agreement(X, Y) is expressed as follows:

[0054]

[0055] In formula (5), p Y s X Power of;

[0056] p represents a prime number pre-agreed upon by each client;

[0057] %p represents the modulo operation on a prime number p.

[0058] Preferably, in step S43, the gradient is updated using the following formula (6).

[0059]

[0060] In formula (6), a(X, Y) represents 1 or -1. Clients are numbered as {1, 2, ..., X, ..., N}. If the number of client X is greater than the number of client Y, then the value is equal to 1; otherwise, it is equal to -1.

[0061] ∑ Y∈{1，2，...，N}\{X} This represents the summation of all non-X clients Y with respect to the notation Y.

[0062] Preferably, in step S4, the summary is performed. The method is expressed by the following formula (7):

[0063]

[0064] In step S4, the method for updating the embedding matrix I is expressed by the following formula (8):

[0065]

[0066] In formula (8), λ I The regularization parameter represents the embedding matrix I.

[0067] Preferably, the gradient generated in step S3 After adding noise, proceed to step S4 to process the gradient. The method of adding noise is expressed by the following formula (9):

[0068]

[0069] In formula (9), n X This represents Gaussian noise.

[0070] The present invention has the following beneficial effects:

[0071] 1. Utilizing the gradient of secure aggregation and Obtaining training samples for the recommendation model ensures that user data does not leave the local machine, while also making the recommendation model training process more secure.

[0072] 2. By masking and adding noise to the gradient, the leakage of source data information is effectively avoided due to the exposure of the true gradient;

[0073] 3. The gradient aggregation method based on secure aggregation provided has lower computational complexity and faster computation speed for gradient encryption and decryption compared to the homomorphic encryption technology used in the background technology, which is conducive to improving the training speed of the recommendation model.

[0074] 4. The recommendation model is trained based on the matrix factorization algorithm provided in this application under the federated learning framework. During the model training process, the participants do not need to exchange local data, which more effectively ensures that local data is not leaked. Attached Figure Description

[0075] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0076] Figure 1 This is a diagram illustrating the implementation steps of a matrix factorization method based on secure aggregation and key exchange according to an embodiment of the present invention.

[0077] Figure 2 This is a flowchart of a matrix factorization method based on secure aggregation and key exchange provided in an embodiment of the present invention. Detailed Implementation

[0078] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0079] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual images. They should not be construed as limiting the scope of this patent. To better illustrate the embodiments of the present invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0080] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "inner," and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present patent. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0081] In the description of this invention, unless otherwise explicitly specified and limited, the term "connection" or similar designation indicating a connection between components should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral part; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication between two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0082] The following example, using three clients (A, B, and C), illustrates how the matrix factorization method based on secure aggregation and key exchange provided in this embodiment is implemented:

[0083] In the federated learning framework, the scheduler is the server, each participant in the training is the client, M is the rating matrix (e.g., the matrix of movie ratings from multiple users on IMDb, containing some missing terms that need to be predicted and filled), and U... A U B U C Let A represent the embedding matrix of local users for clients A, B, and C (using matrices to quantify local users), and let I represent the embedding matrix of items (using matrices to quantify shared items). For example... Figure 2As shown, the specific implementation steps of the matrix factorization method based on secure aggregation and key exchange provided in this embodiment are as follows:

[0084] 1. All parties determine the embedding dimension (the embedding dimension represents how many dimensions of space are used to quantify users and items). The server initializes the item embedding matrix I according to the embedding dimension, and clients A, B, and C initialize their own local user embedding matrices U according to the embedding dimension. A U B U C

[0085] 2. The server broadcasts the embedded matrix I to clients A, B, and C;

[0086] 3. Client A calculates U using the embedding matrix I. A gradient Then update the local user's embedding matrix U. A , in m A I represents the total number of users in client A. j This represents the embedding vector of item j that is common to all clients. Indicate I j The vector transpose, This represents the rating of user i for item j held by client A. This indicates the items j that user i actually rated, owned by client A. This represents the summation of the items j that user i actually rated with respect to the symbol i, owned by client A; U A The update method is as follows: λ U Show U A The regularization parameter;

[0087] The gradients corresponding to clients B and C respectively The calculation method, and the update of U respectively B U C The method is the same as that used by client A, and will not be repeated here;

[0088] 4. Client A uses the locally updated U A Calculate the gradient generated by the user with respect to the embedding matrix I. in d represents the total number of items in total. This refers to users i owned by client A who have rated item j. Sum the values ​​of user i who have rated item j for client A with respect to the symbol i.

[0089] The gradients corresponding to clients B and C respectively The calculation method is the same as that for client A, and will not be repeated here;

[0090] To avoid exposing the true gradient, it is preferable to add noise to the gradient corresponding to each client. More preferably, differential privacy technology is used to isolate the gradients of clients A, B, and C. Add Gaussian noise n respectively A n B n C Taking client A as an example, n A Represents the generated random matrix that follows a Gaussian distribution (size and...) same), Updated to

[0091] 5. Clients A, B, and C generate their own public and private keys locally, p A p B p C These represent the public keys generated locally by clients A, B, and C, respectively. A s B s C These represent the private keys generated locally by clients A, B, and C, respectively. Taking client A as an example, the private key s... A A locally generated random number (numerically less than p), p A (via private key) A (calculation) is Where g represents the generator (a primitive root modulo p, which can be a small number, such as 2 for simplicity). s represents g A The power of p is a large prime number (usually up to 2048 bits). %p represents the modulo operation on p. The values ​​of g and p are predetermined for each client.

[0092] 6. The server collects all public keys p A p B p C The public key p is sent to client A. B p C The public key sent to client B is p A p C The public key sent to client C is p A p B ;

[0093] 7. Client A uses public key p B p C and locally generated private key s AGenerate a key_agreement(A, B) with client B and a key_agreement(A, C) with client C; client B uses the public key p A p C and private key B Generate a key_agreement(A, B) with client A and a key_agreement(B, C) with client C; client C uses the public key p A p B and your private key C Generate a key_agreement(A, C) with client A and a key_agreement(B, C) with client B. Taking client A as an example... They represent p respectively B s A Power and p C s A The power of p is represented by %p, which means taking the modulo operation on p.

[0094] 8. Client A uses its local key_agreement(A, B) as a seed to generate mask mask(A, B), and its local key_agreement(A, C) as a seed to generate mask mask(A, C), and updates the gradient. Client B uses its local key_agreement(A, B) as a seed to generate mask mask(A, B), and its local key_agreement(B, C) as a seed to generate mask mask(B, C), and updates the gradient. Client C uses its local key_agreement(A, C) as a seed to generate mask mask(A, C), and its local key_agreement(B, C) as a seed to generate mask mask(B, C), and updates the gradients.

[0095] Taking client A as an example, the mask mask(A, B) is generated using key_agreement(A, B) as the seed, and its size, shape, and... The same random matrix (can be generated directly by calling open-source library functions and passing in the seed parameter).

[0096] 9. The server summarizes the gradients to obtain... Then update I to get λ I Represents the regularization parameter of the embedding matrix I;

[0097] 10. Repeat steps 2-8 until the maximum number of training iterations of the federated recommendation model is reached or the algorithm converges.

[0098] In short, the matrix factorization method based on secure aggregation and key exchange provided in this embodiment, such as Figure 1 As shown, the steps include:

[0099] S1, let the scheduler of the federated learning framework be the server, and each participant in the training be the client. The server will broadcast the initialized item embedding matrix I to each client.

[0100] S2, Each client X uses the embedding matrix I to calculate the embedding matrix U of its local user. X gradient and utilize Update the local user's embedded matrix U X ;

[0101] S3, each client X utilizes the locally updated U X Calculate the gradient generated with respect to the embedding matrix I.

[0102] S4, update the gradient using a key exchange method. And on Summarize to obtain Then, using Update the embedding matrix I;

[0103] S5. Repeat steps S2-S4 until the termination condition of federated learning is met.

[0104] In summary, this invention provides a new approach to enhancing data security in federated learning by safely aggregating the gradients of the item matrix I obtained from matrix factorization within a federated learning framework; it utilizes the safely aggregated gradients... and Obtaining training samples for the recommendation model (i.e., federated learning model) ensures that user data does not leave the local machine, while making the recommendation model training process more secure. Masking and noise-reducing gradients effectively prevents the leakage of source data information due to the exposure of the true gradients. The gradient aggregation method based on secure aggregation provides lower computational complexity and faster computation speed for gradient encryption and decryption compared to the homomorphic encryption technology used in the background technology, which is beneficial to improving the training speed of the recommendation model.

[0105] It should be stated that the above-described specific embodiments are merely preferred embodiments of the present invention and the technical principles employed. Those skilled in the art should understand that various modifications, equivalent substitutions, and variations can be made to the present invention. However, such variations, as long as they do not depart from the spirit of the present invention, should be within the scope of protection of the present invention. Furthermore, some terminology used in this specification and claims is not limiting, but merely for ease of description.

Claims

1. A matrix decomposition method based on secure aggregation and key exchange, characterized by the steps of Comprising: S1, let the scheduler of the federated learning framework be the server, and each participating training party be the client. The server initializes the item embedding matrix according to the embedding dimension. And the initialization of the item's embedding matrix The broadcast is sent to each of the clients, and the embedded dimension indicates how many dimensions of space are used to quantify users and items; S2, each client with the embedding matrix computing gradients with respect to the respective local user embedding matrix and updating the local user's embedding matrix with ;​ S3, each of the clients with the locally updated , compute the gradient of the generated gradient ; S4, the client The server updates the gradient by using the key exchange method , and The results are summarized as Then, the embedding matrix is updated by using ;​ S5, repeating steps S2-S4 until a termination condition of federated learning is reached; In step S3, the embedding matrix embedding vector corresponding gradient is calculated by the following equation (3): In equation (3), the first row; an embedding vector representing the item common to all the clients a vector transpose of the embedding vector a vector transpose of the embedding vector representing the embedding matrix of the associated local user in the embedding vector; representing said client said user about said item a rating; representing the client owned by the client those users who have rated behavior ; representing the clients owning items those users that have rated behavior about the token summing.

2. The matrix decomposition method based on secure aggregation and key exchange of claim 1, wherein, In step S4, the aggregation The method of the present application is expressed by the following equation (7): In step S4, the embedding matrix is updated The method is expressed by the following equation (8): In equation (8), denotes the embedding matrix regularization parameter.

3. The matrix decomposition method based on security aggregation and key exchange of claim 1, wherein, said gradient generated in step S3 is added with noise said gradient is added with noise and then proceeds to step S4 The method of adding noise is expressed by the following equation (9) In equation (9), represents a Gaussian noise.

Images