A face recognition method based on secret sharing and federated learning

Abstract

The application provides a face recognition method based on secret sharing and federated learning, and relates to the technical field of face recognition. The method comprises a secret sharing component and a federated learning component. For unconstrained face pictures, first, in the model training process, the federated learning component is used to train a feature extractor and a face classifier, when the client uploads the model parameters to the server, the differential privacy method is used to add noise to the parameters; at the same time, the distribution algorithm of the secret sharing component is used to distribute the face image into multiple shadow images and store them in a shadow database; secondly, in the face recognition process, the trained feature extractor and classifier of the federated learning component are used to find the face id with the highest similarity, search the corresponding shadow image in a certain number of shadow databases, and restore the face image according to the recovery algorithm of the secret sharing component and return the result. The application realizes the requirements of privacy security and face recognition availability for face recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face recognition, and particularly relates to a face recognition method based on secret sharing and federated learning. BACKGROUND

[0002] Face recognition is the most widely used biometric authentication technology at present, and is applied to Internet finance, information application, production monitoring, access control and other fields closely related to people's production and life. The process of face recognition can be divided into two parts: model training process and recognition process. The model training process includes the training of the feature extractor and the training of the classifier. The recognition process will use the trained model to perform face matching, and display the face pictures stored in the database on the client.

[0003] The main difficulties of face recognition include the requirements of accuracy and security. Existing methods mostly focus on ensuring accuracy, while ignoring the requirement of security. In traditional face recognition systems, whether using centralized storage or distributed storage, when the face pictures are stored in the database, the photos as files are complete. Therefore, when an attacker attacks a database, a part of the complete face information of a person will be leaked. At the same time, a large amount of data is needed for face recognition training to enable the neural network to learn more face features and better distinguish impostors. However, face information as private information cannot be collected in large quantities. In addition, the parameters of the face recognition model are vulnerable to differential attacks when transmitted, resulting in personal information leakage. Therefore, a biometric authentication prototype system with privacy security and face recognition availability is needed. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a face recognition method based on secret sharing and federated learning to realize safer face recognition in view of the shortcomings of the prior art.

[0005] To solve the above technical problems, the technical scheme adopted by the present application is: a face recognition method based on secret sharing and federated learning. For unconstrained face pictures, first, a model is trained using the federated learning method, the server sends the initialization model to the client participating in this round of communication, the client uses the local data set to train the model and adds noise to the model parameters using the differential privacy method, all clients participating in this round of communication upload the model parameters with noise to the server, and the server aggregates the model to form a federated model as the initialization model for the next round of communication, wherein the client is a node with non-shareable face images and computing power. At the same time, the distribution algorithm of secret sharing is used to distribute the face image into multiple shadows and store it in the shadow database. Secondly, in the face recognition process, the trained federated model is used for inference to find the face id with the highest similarity, and the face id is used to search for the corresponding shadow image in a certain number of shadow databases, and then the face image is restored using the recovery algorithm of secret sharing and echoed. Specifically, the following steps are included:

[0006] Step 1: Use the federated learning and differential privacy method to train the face recognition model with the original unconstrained face image as the input, the face recognition model includes a feature extractor and a face classifier;

[0007] Step 1.1: Model initialization;

[0008] Step 1.1.1: Select a federated learning algorithm;

[0009] Step 1.1.2: Select federated learning parameters, set the total number of clients K, the proportion of clients participating in communication per round C, the total number of communication rounds T, and the local training rounds E;

[0010] Step 1.1.3: Select a face recognition model, including a feature extractor and a face classifier;

[0011] Step 1.2: Model training;

[0012] Step 1.2.1: Server distributes the model, the server selects C·K clients from K clients participating in this round of communication, and sends the current federated model as the initialization model to the selected clients;

[0013] Step 1.2.2: Client trains the model;

[0014] Step 1.2.2.1: Each client participating in this round of communication trains the feature extractor using the local private face image as input, and then trains the face classifier using the face feature calculated by the feature extractor as input;

[0015] Step 1.2.2.2: After the local training of E rounds, a certain size of noise is added to each parameter of the model, and the model parameter with noise is uploaded to the server;

[0016] Step 1.2.3: Server aggregates the model;

[0017] The server collects the model parameters of the selected C·K clients, calculates the average value of each parameter in the model, and updates the federated model;

[0018] Step 1.2.4: Repeat steps 1.2.1 to 1.2.3 until T rounds of communication;

[0019] Step 1.3: The server transmits the final federated model to all clients.

[0020] Step 2: Using the distribution algorithm of Shamir secret sharing, each client distributes all local face images into multiple shadow images and stores them in a shadow database;

[0021] Step 2.1: Secret sharing initialization: set parameters n, parameter k and prime number p, n represents that 1 face image is distributed into n shadow images, k represents that at least k shadow images are needed to restore the original face image;

[0022] Step 2.2: Distribute each face image of each client;

[0023] Step 2.2.1: On the RGB three channels, convert each channel of the image into a matrix, each matrix records the pixel value of the image on the channel, and a face image converts three matrices;

[0024] Step 2.2.2: Generate shadow images;

[0025] Step 2.2.2.1: Construct a polynomial, for each pixel of each channel, take the pixel value as a secret, generate k-1 random numbers, and construct a k-1 order polynomial with the secret as the constant term. Substitute {1, 2, …, n} into the polynomial to calculate the n shadow values of the pixel point on a single channel; the secret means that only the client who owns the face image knows the pixel value at this position, and the pixel value is not shared with other clients.

[0026] Step 2.2.2.2: Construct a shadow matrix, calculate the n shadow values of each pixel on each channel in turn, form n shadow matrices on each channel, and form 3*n shadow matrices in total;

[0027] Step 2.2.2.3: Construct a shadow image, convert the corresponding three shadow matrices on the three channels into a shadow image, and form n shadow images in total;

[0028] Step 2.3: Store the shadow images, store n shadow images into n different shadow databases respectively, and delete the original face image information.

[0029] Step 3: Use the trained federated model to perform face recognition;

[0030] Step 3.1: The client inputs the face image to be identified into the feature extractor part of the federated model obtained in step 2, and outputs the features of the face.

[0031] Step 3.2: The client inputs the face features into the face classifier part of the federated model obtained in step 1, and outputs the id of the face.

[0032] Step 4: Use the recovery algorithm of Shamir secret sharing to restore the original face image from k shadow images;

[0033] Step 4.1: The client randomly selects k shadow databases from the n shadow databases; in each database, the corresponding shadow image is retrieved using the id obtained in step 3, and a total of k shadow images are obtained;

[0034] Step 4.2: Construct a face image matrix;

[0035] Step 4.2.1: In the RGB three channels, convert each channel of the k shadow images obtained in 4.1 into a matrix, each matrix records the pixel value of the shadow image in that channel, and each shadow image converts three matrices;

[0036] Step 4.2.2: Obtain the pixel values of the k shadow images at the same position in the same channel, reconstruct the k-1 polynomial using the k pixel values, and input 0 into the polynomial for calculation. The calculation result is the single-channel pixel value of the original face image at that position;

[0037] Step 4.2.3: Restore all pixel values of each channel of the face image in turn to obtain the face image matrix;

[0038] Step 4.3: Concatenate the three face image matrices obtained in 4.2 into a three-channel face image, and echo the restored face image on the client.

[0039] The beneficial effects produced by the technical scheme are that the face recognition method based on secret sharing and federated learning is provided.The secret sharing method is used to segment the pictures, and the pictures are stored in multiple databases, so that even if some databases are leaked, the face pictures will not be leaked; the federated learning method is used to train the face recognition model, so that the model can be trained without collecting the face pictures, and even if a training set is attacked, the federated model will not be leaked; the differential privacy technology is used to add noise to the client model parameters, so that even if the model parameters uploaded by the client are subjected to differential attack, the attacker cannot determine whether a user participates in the training from the model parameters. The privacy protection security is improved under the premise of ensuring the availability of face recognition, and the application has important value for the real environment. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 The total architecture diagram of the face recognition method provided by the embodiment of the present application is shown in the figure.

[0041] Figure 2 The architecture diagram of the secret sharing component provided by the embodiment of the present application is shown in the figure.

[0042] Figure 3 The architecture diagram of the federated learning component provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0043] The specific embodiments of the present application are described in detail below in combination with the drawings and examples. The following examples are used to illustrate the present application, but are not used to limit the scope of the present application.

[0044] The specific process is shown in the figure. Figure 1

[0045] Step 1: using the method of federated learning and differential privacy, taking the original unconstrained face image as the input, training the face recognition model, the face recognition model including the feature extractor and the face classifier;

[0046] Step 1.1: model initialization;

[0047] Step 1.1.1: selecting the federated learning algorithm, using the FedAvg algorithm for federated learning;

[0048] Step 1.1.2: selecting the federated learning parameters, setting the total number of clients K, the proportion of clients participating in communication per round C, the total number of communication rounds T, and the local training rounds E respectively;

[0049] ​Step 1.1.3: Select a face recognition model, use Sphere as a feature extractor, and use CosFace as a face classifier;

[0050] Step 1.2: Model training, the training process is as shown in Figure 3

[0051] Step 1.2.1: Server distributes the model, the server selects C·K clients from K clients to participate in this round of communication, and sends the current federated model as an initialization model to the selected clients;

[0052] Step 1.2.2: Client trains the model;

[0053] Step 1.2.2.1: Each client participating in this round of communication trains the feature extractor by inputting the local private face image; then trains the face classifier by inputting the face image feature calculated by the feature extractor;

[0054] Step 1.2.2.2: After E rounds of local training, add a certain size of Laplace noise to each parameter of the model, and upload the model parameters with noise to the server;

[0055] Step 1.2.3: Server aggregates the model;

[0056] The server collects the model parameters of the selected C·K clients, calculates the average value of each parameter in the model, and updates the federated model;

[0057] Step 1.2.4: Repeat steps 1.2.1 to 1.2.3 until T rounds of communication;

[0058] Step 1.3: Server transmits the final federated model to all clients.

[0059] Step 2: Use the Shamir secret sharing distribution algorithm, each client distributes all local face images into multiple shadow images and stores them in a shadow database, the distribution process is as shown in Figure 2 the left half;

[0060] Step 2.1: Secret sharing initialization: set parameters n and k respectively, let prime number p = 257, n represents distributing 1 face image into n shadow images, and k represents that at least k shadow images are needed to recover the original face image; In an image I with resolution (h, w), h represents the image height and w represents the image width;

[0061] Step 2.2: Distribute each face image of each client;

[0062] ​Step 2.2.1: On the RGB three channels, convert each channel of the image into a matrix, each matrix records the pixel value of the image on the channel, and a face image converts three matrices; let I c be the matrix of the face image on channel c, and c = {0, 1, 2} then I0is the matrix of the face image on the R channel, I1is the matrix of the face image on the G channel, and I2is the matrix of the face image on the B channel; wherein the pixel value of the i-th row and j-th column of the matrix I c is expressed as I c,i,j ;

[0063] Step 2.2.2: Generate shadow images;

[0064] Step 2.2.2.1: Construct a polynomial, for each pixel of each channel, take the pixel value as a secret, generate k-1 random numbers, and construct a k-1 order polynomial with the secret as a constant term, and calculate n shadow values of the pixel on a single channel by substituting {1, 2, …, n} into the polynomial:

[0065]

[0066] Where F c,i,j (x) is a k-1 order polynomial constructed with I c,i,j as a constant term, and x ∈ {1, 2, …, k-1}; is the l-th random number generated randomly, and l ∈ {1, 2, …, k-1}; then the n shadow values generated by I c,i,j as a secret are

[0067] {F c,i,j (1), F c,i,j (2), …, F c,i,j (n)};

[0068] Step 2.2.2.2: Construct a shadow matrix, calculate the n shadow values of each pixel on each channel in turn, form n shadow matrices on each channel, and form 3*n shadow matrices in total:

[0069]

[0070] Where SI x is the x-th shadow image, then is the x-th shadow matrix on channel c, is the shadow value of the i-th row and j-th column.

[0071] Step 2.2.2.3: Construct a shadow image, convert the corresponding three shadow matrices on the three channels into a shadow image, and form n shadow images in total:

[0072]

[0073] Step 2.3: Store the shadow images, each client has n shadow databases, SI x is stored in the shadow database SDB x of the client, and the original face image information is deleted, where SDB x represents the xthshadow database of the client who owns the face image I;

[0074] Step 3: Face recognition using the trained federated model;

[0075] Step 3.1: The client inputs the face image to be identified into the feature extractor part of the federated model obtained in step 2, and outputs the features of the face;

[0076] Step 3.2: The client inputs the face features into the face classifier part of the federated model obtained in step 2, and outputs the id of the face.

[0077] Step 4: Restore the original face image from k shadow images using the Shamir secret sharing restoration algorithm, the restoration process is shown in the right half of the figure; Figure 2

[0078] Step 4.1: The client randomly selects k shadow databases from the n shadow databases, Q = {q m |m = 1, 2, …, k} is the number of the k shadow databases selected by the client, q m The corresponding shadow database is In each database, the shadow image corresponding to the id is retrieved from using the id obtained in step 3. The k shadow images obtained are

[0079] Step 4.2: Construct the face image matrix;

[0080] Step 4.2.1: In the RGB three channels, convert each channel of the k shadow images obtained in 4.1 into a matrix, each matrix records the pixel value of the shadow image in that channel, and a shadow image converts three matrices:

[0081]

[0082] Step 4.2.2: Obtain the pixel values of the same position in the same channel of the k shadow images, reconstruct the k-1 polynomial using the k pixel values, input 0 into the polynomial for calculation, and the calculation result is the single-channel pixel value of the original face image at that position:​

[0083]

[0084] wherein let be the restored face image, then be the face image matrix restored on the channel c, represent the pixel value restored on the i-th row and j-th column.

[0085] Step 4.2.3: restore all pixel values of each channel of the face image in turn, and the three face image matrices obtained are

[0086] Step 4.3: splice the three face image matrices obtained in 4.2 into a three-channel face image, and echo the restored face image on the client:

[0087]

[0088] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope defined by the claims of the present application.

Claims

1. A face recognition method based on secret sharing and federated learning, characterized in that, The method specifically includes the following steps: Step 1: Using federated learning and differential privacy methods, train a face recognition model with the original unconstrained face image as input. The face recognition model includes a feature extractor and a face classifier. Step 2: Using Shamir's secret sharing distribution algorithm, each client distributes all local face images into multiple shadow images and stores them in the shadow database; Step 2.1: Secret Sharing Initialization: Set parameters respectively ,parameter and prime numbers , This means distributing one face image as Zhang's shadow image, It means at least Only by extracting a shadow image can the original face image be recovered; Step 2.2: Distribute each face image to each client; Step 2.2.1: On the RGB three channels, each channel of the image is converted into a matrix. Each matrix records the pixel value of the image on that channel. A face image is converted into three matrices. Step 2.2.2: Generate the shadow image; Step 2.2.2.1: Construct a polynomial. For each pixel in each channel, use the pixel value as a secret to generate... A set of random numbers, with the secret term as a constant. A polynomial of order 1 will Substituting the values ​​into the polynomial, we can calculate the pixel's position on a single channel. A shadow value; the secret refers to the fact that only the client that possesses the face image knows the pixel value at that location, and this pixel value is not shared with other clients; Step 2.2.2.2: Construct the shadow matrix, and calculate the shadow value for each pixel in each channel sequentially. Each channel forms a shadow value. A total of shadow matrices are formed, forming a total of 10 shadow matrices. One shadow matrix; Step 2.2.2.3: Construct shadow images by converting the three corresponding shadow matrices on the three channels into shadow images, thus forming a total of Zhang's shadow image; Step 2.3: Store the shadow image. Each shadow image is stored in Remove the original face image information from a different shadow database; Step 3: Perform face recognition using the trained federated model; Step 4: Using the Shamir secret sharing recovery algorithm, The shadow image was restored to the original face image.

2. The face recognition method based on secret sharing and federated learning according to claim 1, characterized in that, Step 1 specifically includes the following steps: Step 1.1: Model initialization; Step 1.1.1: Select a federated learning algorithm; Step 1.1.2: Select federated learning parameters and set the total number of clients respectively. The proportion of clients participating in each round of communication Total number of communication rounds Local training rounds ; Step 1.1.3: Select a face recognition model, including a feature extractor and a face classifier; Step 1.2: Model training; Step 1.2.1: Server distribution model, the server from Selecting clients from among those clients to participate in this round of communication One client, and send the current federated model as the initialization model to the selected client; Step 1.2.2: Client-side model training; Step 1.2.2.1: Each client participating in this round of communication uses its local private face image as input to train a feature extractor; then, the face image features calculated by the feature extractor are used as input to train a face classifier. Step 1.2.2.2: Local Training After the round, a certain amount of noise is added to each parameter of the model, and the model parameters with added noise are uploaded to the server; Step 1.2.3: Server aggregation model; The server collects the selected data. The model parameters of each client are calculated, the average value of each parameter in the model is calculated, and the federated model is updated. Step 1.2.4: Repeat steps 1.2.1 to 1.2.3 until communication is established. wheel; Step 1.3: The server delivers the final federated model to all clients.

3. The face recognition method based on secret sharing and federated learning according to claim 1, characterized in that, Step 3 specifically includes the following steps: Step 3.1: The client inputs the face image to be recognized into the feature extractor part of the federated model obtained in step 2, and outputs the features of the face; Step 3.2: The client inputs the facial features into the face classifier part of the federated model obtained in Step 1, and outputs the ID of the face.

4. The face recognition method based on secret sharing and federated learning according to claim 1, characterized in that, Step 4 specifically includes the following steps: Step 4.1: The client starts from... Randomly select from the shadow database A shadow database; in each database, the corresponding shadow image is retrieved using the ID obtained in step 3, resulting in a total of [number missing] shadow images. Zhang's shadow image; Step 4.2: Construct the face image matrix; Step 4.3: Concatenate the three face image matrices obtained in step 4.2 into a three-channel face image, and display the restored face image on the client.

5. The face recognition method based on secret sharing and federated learning according to claim 4, characterized in that, Step 4.2 specifically includes the following steps: Step 4.2.1: On the three RGB channels, the values ​​obtained in step 4.1 are... Each channel of a shadow image is converted into a matrix, and each matrix records the pixel value of the shadow image in that channel. A shadow image is converted into three matrices in total. Step 4.2.2: Obtain The pixel values ​​of the shadow images at the same location on the same channel, using this Reconstructing pixel values The polynomial of degree 1 is calculated by inputting 0 into the polynomial. The result is the single-channel pixel value of the original face image at that location. Step 4.2.3: Sequentially recover all pixel values ​​of each channel of the face image to obtain the face image matrix.

Images