Face recognition model training method and device based on federated learning
By using the equivalent category vector matrix and cosine similarity loss function in federated learning, the problems of insufficient privacy protection and low recognition accuracy in single-device face recognition model training are solved, and the accuracy of face recognition is improved while protecting privacy.
Patent Information
- Application Number
- CN202111509411.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-10
- Publication Date
- 2025-10-21
- Estimated Expiration
- 2041-12-10
AI Technical Summary
Existing federated learning methods have problems with insufficient privacy protection and low recognition accuracy in face recognition model training. Especially when there is only one type of data on a single device, it is difficult to effectively use large-scale face data for training.
By receiving the equivalent category vector matrix and backbone network parameters sent by the server, combined with the sample face images and category vectors of the target client, the face recognition model is trained. A loss function based on cosine similarity is used to ensure that the category vectors are not shared, thereby improving the recognition accuracy.
While protecting user privacy, the accuracy of face recognition is improved, the leakage of category vectors is avoided, and the recognition effect of the model is enhanced.
Smart Images
Figure CN114360002B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a face recognition model training method and device based on federated learning. Background Art
[0002] Facial recognition is a biometric technology that identifies people based on their facial features. Generally speaking, the more training data a model has, the better its recognition performance. However, facial data is considered private information. In real life, it is common for a device to only have one category. Therefore, obtaining large-scale facial data to train facial recognition networks while protecting facial privacy is a challenge.
[0003] Federated learning is a machine learning model based on datasets distributed across multiple devices, while prohibiting data exchange between devices to prevent data leakage. Currently, methods for storing only one type of data on a device and not exchanging category vectors between devices include Federated Averaging with Spreadout (FedAwS) and Federated User Verification (FedUV). In the FedAwS method, the server receives the category vectors sent by the client and uses a regularization term to distance the client's category vectors from each other; in the FedUV method, error-correcting codes are used to maximize the distance between the category vectors of the client. However, the face recognition models obtained by the FedAwS and FedUV methods still lag significantly behind the performance of face recognition models trained centrally using deep learning methods. Summary of the Invention
[0004] The present invention provides a face recognition model training method and device based on federated learning, which are used to solve the problem of low face recognition accuracy in the prior art and improve the accuracy of face recognition while protecting user privacy.
[0005] In a first aspect, the present invention provides a face recognition model training method based on federated learning, which is applied to a target client, and the method includes:
[0006] Receiving an equivalent category vector matrix and a first backbone network parameter sent by a server; the equivalent category vector matrix includes: n equivalent category vectors; the n equivalent category vectors are obtained by the server performing a fusion calculation on the category vectors of Cm candidate clients;
[0007] Training a face recognition model based on the first backbone network parameters, the sample face image of the target client, the first category vector of the target client, and the equivalent category vector matrix to obtain a target face recognition model and a second category vector of the target client;
[0008] Sending the second category vector and the target backbone network parameters of the target face recognition model to the server.
[0009] According to a face recognition model training method based on federated learning provided by the present invention, the loss function of the target face recognition model is determined based on the cosine similarity between the feature vector of the sample face image and the first category vector of the target client, and the cosine similarity between the feature vector of the sample face image and the first category vector and the equivalent category vector matrix.
[0010] According to a face recognition model training method based on federated learning provided by the present invention, the loss function is expressed by formula (1):
[0011]
[0012] Wherein, N represents the number of sample face images of the target client; represents the cosine similarity between the category vector of the i-th sample and the feature vector of the i-th sample, i represents the sample of the client, y i is the category shown by the i-th sample; s j,i Represents the category vector w j and the cosine similarity between the feature vector of the i-th sample, j represents the category vector of each client; C refers to the sum of the number of category vectors of each client and the number of equivalent category vectors.
[0013] According to a face recognition model training method based on federated learning provided by the present invention, before receiving the equivalent category vector matrix and the first backbone network parameters sent by the server, the method further includes:
[0014] Sending a first identifier of the target client to the server; the first identifier is used to represent the target client.
[0015] In a second aspect, the present invention further provides a face recognition model training method based on federated learning, which is applied to a server, and the method comprises:
[0016] Performing a fusion calculation on the category vectors of Cm candidate clients among the C candidate clients to obtain an equivalent category vector matrix; wherein the equivalent category vector matrix includes n equivalent category vectors;
[0017] Sending the first backbone network parameter and the equivalent category vector matrix to m candidate clients other than the Cm candidate clients among the C candidate clients respectively;
[0018] Receiving the target backbone network parameter and the second category vector sent by each candidate client among the m candidate clients;
[0019] Aggregate the target backbone network parameters sent by each of the m candidate clients to obtain second backbone network parameters.
[0020] According to a face recognition model training method based on federated learning provided by the present invention, the category vectors of Cm candidate clients among the C candidate clients are fused and calculated to obtain an equivalent category vector matrix, including:
[0021] Randomly select n pairs of clients from the Cm candidate clients, perform fusion calculation on the category vectors of each pair of clients in the n pairs of clients, and obtain n equivalent category vectors.
[0022] According to a face recognition model training method based on federated learning provided by the present invention, the fusing calculation of the category vectors of each pair of clients in the n pairs of clients to obtain n equivalent category vectors includes:
[0023] The category vectors of each pair of clients in the n pairs of clients are respectively fused and calculated using formula (2) and formula (3) to obtain n equivalent category vectors;
[0024]
[0025]
[0026] Among them, the w i and w j Represents the category vector of each pair of clients; For the w i and w j The corresponding equivalent category vector.
[0027] In a third aspect, the present invention further provides a face recognition model training device based on federated learning, which is applied to a target client, and the device comprises:
[0028] A first receiving module is configured to receive an equivalent category vector matrix and a first backbone network parameter sent by a server; the equivalent category vector matrix includes n equivalent category vectors; the n equivalent category vectors are obtained by the server performing a fusion calculation on category vectors of Cm candidate clients;
[0029] a training module, configured to train a face recognition model based on the first backbone network parameters, a sample face image of the target client, the first category vector of the target client, and the equivalent category vector matrix, to obtain a target face recognition model and a second category vector of the target client;
[0030] The first sending module is used to send the second category vector and the target backbone network parameters of the target face recognition model to the server.
[0031] In a fourth aspect, the present invention further provides a face recognition model training device based on federated learning, which is applied to a server, and the device comprises:
[0032] A fusion module, configured to perform fusion calculation on the category vectors of Cm candidate clients among the C candidate clients to obtain an equivalent category vector matrix; wherein the equivalent category vector matrix includes n equivalent category vectors;
[0033] A second sending module is configured to send the first backbone network parameter and the equivalent category vector matrix to m candidate clients other than the Cm candidate clients among the C candidate clients respectively;
[0034] A second receiving module is configured to receive the target backbone network parameters and the second category vector sent by each of the m candidate clients;
[0035] The aggregation module is configured to aggregate the target backbone network parameters sent by each of the m candidate clients to obtain second backbone network parameters.
[0036] In a fifth aspect, the present invention also provides a target client, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the steps of the face recognition model training method based on federated learning as described in the first aspect are implemented.
[0037] In the sixth aspect, the present invention also provides a server, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the steps of the face recognition model training method based on federated learning as described in any one of the second aspects are implemented.
[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the steps of the face recognition model training method based on federated learning as described in any one of the first aspects above, or the steps of the face recognition model training method based on federated learning as described in any one of the second aspects above.
[0039] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the face recognition model training method based on federated learning as described in any one of the first aspects above, or the steps of the face recognition model training method based on federated learning as described in any one of the second aspects above.
[0040] The face recognition model training method based on federated learning provided by the present invention receives an equivalent category vector matrix and first backbone network parameters sent by a server, uses the first backbone network parameters, a sample face image of a target client, a first category vector of the target client and an equivalent category vector matrix to train the face recognition model of the target client, obtains a trained target face recognition model of the target client and a second category vector of the target client, and sends the second category vector and target backbone network parameters of the target face recognition model to the server. Through the equivalent category vector, the category vector is not shared between the target clients, thereby ensuring the user's privacy information and improving the accuracy of user face recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0042] Figure 1 This is one of the flow charts of the face recognition model training method based on federated learning applied to a target client provided by the present invention;
[0043] Figure 2 This is the second flow chart of the face recognition model training method based on federated learning applied to a server provided by the present invention;
[0044] Figure 3 This is the third flow chart of the face recognition model training method based on federated learning applied to a server provided by the present invention;
[0045] Figure 4 This is the fourth flow chart of the face recognition model training method based on federated learning provided by the present invention;
[0046] Figure 5 This is one of the structural diagrams of the face recognition model training device based on federated learning provided by the present invention;
[0047] Figure 6 This is the second structural diagram of the face recognition model training device based on federated learning provided by the present invention;
[0048] Figure 7 It is a structural diagram of the target client provided by the present invention. DETAILED DESCRIPTION
[0049] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.
[0050] The following combination Figures 1-4 The present invention describes a face recognition model training method based on federated learning.
[0051] Figure 1 This is a flow chart of a face recognition model training method based on federated learning applied to a target client provided by the present invention, such as Figure 1 As shown, the method includes:
[0052] Step 110: Receive an equivalent category vector matrix and first backbone network parameters sent by a server; the equivalent category vector matrix includes: n equivalent category vectors; the n equivalent category vectors are obtained by the server performing a fusion calculation on the category vectors of Cm candidate clients.
[0053] Optionally, the federated learning-based face recognition model training method provided by the present invention can be applied to face recognition scenarios. The execution entity of the federated learning-based face recognition model training method provided by the present invention can be the federated learning-based face recognition model training device provided by the present invention, such as a target client or a control module in the federated learning-based face recognition model training device for executing the federated learning-based face recognition model training method.
[0054] Optionally, before the target client receives the equivalent category vector matrix and the first backbone network parameter sent by the server, the first identifier of the target client is sent to the server; wherein the first identifier is used to represent the target client, that is, the server knows the first identifier of the target client in advance.
[0055] It should be noted that the target client is a device that recognizes facial images. For example, the target client can be a mobile phone, laptop computer, tablet computer, or other terminal device. The user can use the target client to verify their facial image, and then perform the corresponding target operation on the target client after the verification is passed.
[0056] Optionally, the equivalent category vector matrix is obtained by combining n equivalent category vectors. The equivalent category vectors can prevent information exchange between target clients. Furthermore, the target client cannot restore its own category vector using the equivalent category vectors, thus ensuring that the target client's category vector information is not leaked. It should be noted that when a target client's category vector moves away from the equivalent category vector during training, the target client's category vector will also move away from the original category vector from which the equivalent category vector is calculated by fusion.
[0057] Optionally, the first backbone network parameters are parameters of a face recognition model.
[0058] Step 120: Training a face recognition model based on the first backbone network parameters, the sample face image of the target client, the first category vector of the target client, and the equivalent category vector matrix to obtain a target face recognition model and a second category vector of the target client.
[0059] Optionally, the first category vector represents the feature vector of the category to which the sample face image of the target client belongs; the second category vector represents the feature vector of the category to which the sample image of the target client belongs after training the face recognition model is completed.
[0060] Optionally, the target face recognition model represents the face recognition model finally obtained after the face recognition model training is completed.
[0061] Optionally, the target client trains the target client's face recognition model based on the first backbone network parameters, equivalent category vector matrix, sample face images in which the target client participates in training, and the first category vector of the target client received from the server; after the training is completed, the target client obtains the target face recognition model and the second category vector of the target client.
[0062] It should be noted that after each round of training, the second category vector of the target client and the first backbone network parameters of the face recognition model will change.
[0063] Step 130: Send the second category vector and the target backbone network parameters of the target face recognition model to the server.
[0064] Optionally, the target backbone network parameters represent parameters corresponding to the target face recognition model.
[0065] The face recognition model training method based on federated learning provided by the present invention receives an equivalent category vector matrix and first backbone network parameters sent by a server, uses the first backbone network parameters, a sample face image of a target client, a first category vector of the target client and an equivalent category vector matrix to train the face recognition model of the target client, obtains a trained target face recognition model of the target client and a second category vector of the target client, and sends the second category vector and target backbone network parameters of the target face recognition model to the server. Through the equivalent category vector, the category vector is not shared between the target clients, thereby ensuring the user's privacy information and improving the accuracy of user face recognition.
[0066] Optionally, the loss function of the target face recognition model is determined based on the cosine similarity between the feature vector of the sample face image and the first category vector of the target client, and the cosine similarity between the feature vector of the sample face image and the first category vector and the equivalent category vector matrix.
[0067] Specifically, the loss function is expressed using formula (1):
[0068]
[0069] Where N represents the number of sample face images of the target client; represents the cosine similarity between the category vector of the i-th sample and the feature vector of the i-th sample, i represents the sample of the client, y i is the category shown by the i-th sample; s j,i Represents the category vector w j and the cosine similarity between the feature vector of the i-th sample, j represents the category vector of each client; C refers to the sum of the number of category vectors of each client and the number of equivalent category vectors.
[0070] It should be noted that when the target client trains the face recognition model based on the first backbone network parameters, the target client's sample face image, the target client's first category vector and the equivalent category vector matrix, in each round of training, the target client obtains the loss value according to the loss function, uses the loss value to judge the training degree of the face recognition model, and updates the face recognition model.
[0071] To determine whether the target client has finished training the face recognition model, the server can collect the loss values of each target client for judgment, or the server can test the face images in the test set and obtain the recognition results of the face images in the test set for judgment, until the accuracy of the face recognition results reaches a stable level and the training is completed.
[0072] Optionally, Figure 2 The second flow chart of the face recognition model training method based on federated learning applied to the server provided by the present invention is as follows: Figure 2 As shown, the method includes:
[0073] Step 210 , performing fusion calculation on the category vectors of Cm candidate clients among the C candidate clients to obtain an equivalent category vector matrix; wherein the equivalent category vector matrix includes n equivalent category vectors.
[0074] Optionally, the server obtains the number of candidate clients for face recognition model training based on the first identifiers of the C candidate clients.
[0075] Specifically, the server initializes the category vectors of C candidate clients, and then selects m candidate clients from the C candidate clients after initialization; the selected m candidate clients are used to train the face recognition model; for the Cm candidate clients, the server's equivalent category vector generator performs a fusion calculation on the category vectors of the Cm candidate clients to obtain an equivalent category vector; then, based on the equivalent category vector, an equivalent category vector matrix is obtained.
[0076] It should be noted that the equivalent category vector matrix includes n equivalent category vectors, which are obtained by stacking n equivalent category vectors.
[0077] Step 220: Send the first backbone network parameters and the equivalent category vector matrix to the m candidate clients other than the Cm candidate clients among the C candidate clients respectively.
[0078] Optionally, while initializing the category vectors of the C candidate clients, the server also initializes the parameters of the first backbone network; after initialization, the server sends the equivalent category vector matrix and the parameters of the first backbone network obtained by fusion calculation of the m candidate clients other than the Cm candidate clients to the m candidate clients.
[0079] Step 230: Receive the target backbone network parameters and the second category vector sent by each of the m candidate clients.
[0080] Optionally, the target backbone network parameters are the parameters of the face recognition model obtained after the face recognition model training is completed for each of the m candidate clients; the second category vector is the category vector of each of the m candidate clients after the face recognition model training is completed for each of the m candidate clients.
[0081] Specifically, after the m candidate clients complete training of the face recognition model, the category vector of each of the m candidate clients is the second category vector, and the parameters of the target face recognition model are the target backbone network parameters; each of the m candidate clients sends the target backbone network parameters and the second category vector to the server, so the server receives the target backbone network parameters and the second category vector sent by each of the m candidate clients.
[0082] Step 240: Aggregate the target backbone network parameters sent by each of the m candidate clients to obtain second backbone network parameters.
[0083] Optionally, the second backbone network parameter is obtained by performing weighted averaging on the target backbone network parameter sent by each candidate client among the m candidate clients after the server receives the target backbone network parameter sent by each candidate client.
[0084] The present invention provides a face recognition model training method based on federated learning. The server obtains an equivalent category vector matrix by fusing the category vectors of Cm candidate clients. The server then sends the equivalent category vector matrix and first backbone network parameters to m candidate clients, respectively, for the m candidate clients to train the face recognition model. After the training is completed, the server receives the target backbone network parameters and second category vectors sent by each of the m candidate clients participating in the training. The server then aggregates the target backbone network parameters sent by each of the m candidate clients to obtain the second backbone network parameters. By using the equivalent category vectors, the category vectors are not shared among the m candidate clients, thereby ensuring the user's privacy and improving the accuracy of user face recognition. At the same time, after the training is completed, the equivalent category vectors do not need to be sent to the server, thus having no impact on the upload bandwidth.
[0085] Optionally, Figure 3 The third flow chart of the face recognition model training method based on federated learning applied to the server provided by the present invention is as follows: Figure 3 As shown, the method includes:
[0086] Step 310 : randomly select n pairs of clients from the Cm candidate clients, perform fusion calculation on the category vectors of each pair of clients in the n pairs of clients, and obtain n equivalent category vectors.
[0087] Optionally, the server selects m candidate clients from C candidate clients to train the face recognition model, and then uses the equivalent category vector generator to randomly select 2 clients from the remaining Cm candidate clients to generate 1 equivalent category vector; the equivalent category vector generator randomly selects n times from the remaining Cm candidate clients, and the equivalent category vector generator selects n pairs of clients, and performs fusion calculation on the category vectors of each pair of clients in the n pairs of clients to obtain n equivalent category vectors.
[0088] Step 320: Send the first backbone network parameters and the equivalent category vector matrix to the m candidate clients other than the Cm candidate clients among the C candidate clients respectively.
[0089] Step 330: Receive the target backbone network parameters and the second category vector sent by each of the m candidate clients.
[0090] Step 340: Aggregate the target backbone network parameters sent by each of the m candidate clients to obtain second backbone network parameters.
[0091] Optionally, the description and explanation of steps 320-340 may refer to the above description and explanation of steps 220-240, and the same technical effect can be achieved. To avoid repetition, they will not be repeated here.
[0092] The present invention provides a method for training a face recognition model based on federated learning. The server randomly selects n pairs of clients from Cm candidate clients, performs a fusion calculation on the category vectors of each pair of clients in the n pairs, and obtains n equivalent category vectors. The server then sends the equivalent category vector matrix and first backbone network parameters to m candidate clients, respectively, for the m candidate clients to train the face recognition model. After the training is completed, the server receives the target backbone network parameters and second category vectors sent by each of the m candidate clients participating in the training. The server then aggregates the target backbone network parameters sent by each of the m candidate clients to obtain the second backbone network parameters. By using equivalent category vectors, the category vectors are not shared among the m candidate clients, thereby ensuring the user's privacy information and improving the accuracy of user face recognition. At the same time, after the training is completed, the equivalent category vectors do not need to be sent to the server, thus having no impact on the upload bandwidth.
[0093] Optionally, step 310 may be implemented in the following manner:
[0094] The category vectors of each pair of clients in n pairs of clients are fused and calculated using formula (2) and formula (3) to obtain n equivalent category vectors;
[0095]
[0096]
[0097] Among them, w i and w j A category vector representing each pair of clients; w i and w j The corresponding equivalent category vector.
[0098] Specifically, the server randomly selects n pairs of clients from Cm candidate clients. For the category vectors of each pair of clients, the server uses formula (2) to calculate the average value of the category vectors of each pair of clients. Then, the server uses formula (3) to calculate the equivalent category vector of the category vector of each pair of clients based on the average value. The category vectors of each pair of clients are calculated n times using formulas (2) and (3) to obtain n equivalent category vectors.
[0099] It should be noted that the client cannot pass the equivalent category vector Restore category vector w i and w j , because any Symmetrical and Two category vectors w on the same semicircle i and w j The equivalent category vector can be obtained by formula (2) and formula (3) Equivalent category vector It ensures that the client information corresponding to the category vector is not leaked. At the same time, in any case, when the client trains its category vector away from the equivalent category vector When , the client's category vector will also be far away from the category vector w i and w j .
[0100] Optionally, Figure 4 The fourth flow chart of the face recognition model training method based on federated learning provided by the present invention is as follows: Figure 4 As shown, the method includes:
[0101] In step 410 , the server initializes the first backbone network parameters and the first category vector of the client; the equivalent category vector generator of the server performs a fusion calculation on the first category vector of the client based on the first category vector of the client to obtain an equivalent category vector matrix.
[0102] In step 420 , the server sends the first backbone network parameter, the first category vector of the client, and the equivalent category vector matrix to the client.
[0103] In step 430, the client receives the first backbone network parameters, the class vector, and the equivalent class vector matrix. The client then trains the face recognition model based on the first backbone network parameters, the client's sample facial image, the client's first class vector, and the equivalent class vector matrix. During training, the client obtains a loss value based on the loss function, determines the training level of the face recognition model based on the loss value, and updates the parameters of the face recognition model.
[0104] In step 440 , the client sends the target backbone network parameters of the face recognition model obtained after the training and the second category vector of the client to the server.
[0105] Step 450: The server receives the target backbone network parameters and the second category vector sent by the client, and aggregates the target backbone network parameters to obtain second backbone network parameters.
[0106] In step 460 , the server sends the second backbone network parameters and the second category vector to the client, so that the client trains the face recognition model again.
[0107] It should be noted that the server receives the target backbone network parameters and the second category vector sent by the client, and aggregates the target backbone network parameters to obtain a global face recognition model; the server uses the face images in the test set to test the global face recognition model corresponding to the second backbone network parameters, and uses two face images for testing each time to obtain the feature vectors corresponding to the two face images output by the global face recognition model. The server compares the similarity of the two face images based on the two feature vectors, and then uses the labels of the two face images for comparison. If the labels are the same, it means that the two face images are the same person, and the accuracy of face image recognition is obtained in the same way; when the accuracy area of face image recognition is stable, the training of the face recognition model is completed, and a trained face recognition model is obtained.
[0108] Optionally, according to Figures 1 to 4 In addition to the description of the face recognition model training method based on federated learning provided by the present invention, the present invention also provides a face recognition method applied to a target client, the method comprising:
[0109] Step 1: Obtain the user's face image.
[0110] Optionally, the face image can be obtained by shooting with a mobile phone, a camera, a video camera, etc., or by reading a photograph that has been taken.
[0111] Step 2: Input the face image into the global face recognition model to obtain the feature vector of the face image output by the global face recognition model; wherein, the global face recognition model is a face recognition model obtained by the server by aggregating the target backbone network parameters of each client.
[0112] Specifically, taking the target client as a mobile phone as an example, a facial image is stored in advance, that is, the facial image is input into the global face recognition model to obtain the feature vector of the facial image. When using the mobile phone for face recognition, the facial image taken by the mobile phone is also input into the global face recognition model to obtain the feature vector of the facial image at this time.
[0113] In step 3, the target client determines the similarity of the feature vectors of the two face images by comparing the feature vectors of the two face images, thereby obtaining the similarity of the two face images.
[0114] Step 4: Compare the similarity between the two facial images with the target threshold. If the similarity between the two facial images is less than the target threshold, it means that the two facial images are not similar and the verification fails. If the similarity between the two facial images is greater than the target threshold, it means that the two facial images are similar and belong to the same person, and the verification succeeds.
[0115] The face recognition model training device based on federated learning provided by the present invention is described below. The face recognition model training device based on federated learning described below and the face recognition model training method based on federated learning described above can be referenced to each other.
[0116] Figure 5 This is a structural diagram of a face recognition model training device based on federated learning provided by the present invention, which is applied to a target client, such as Figure 5 As shown, the face recognition model training device 500 based on federated learning includes: a first receiving module 501, a training module 502 and a first sending module 503; wherein,
[0117] A first receiving module 501 is configured to receive an equivalent category vector matrix and a first backbone network parameter sent by a server; the equivalent category vector matrix includes n equivalent category vectors; the n equivalent category vectors are obtained by the server performing a fusion calculation on category vectors of Cm candidate clients;
[0118] A training module 502 is configured to train a face recognition model based on the first backbone network parameters, a sample face image of the target client, the first category vector of the target client, and the equivalent category vector matrix to obtain a target face recognition model and a second category vector of the target client;
[0119] The first sending module 503 is used to send the second category vector and the target backbone network parameters of the target face recognition model to the server.
[0120] The face recognition model training device based on federated learning provided by the present invention receives an equivalent category vector matrix and a first backbone network parameter sent by a server, and uses the first backbone network parameter, a sample face image of a target client, a first category vector of the target client, and an equivalent category vector matrix to train the face recognition model of the target client, thereby obtaining a trained target face recognition model of the target client and a second category vector of the target client, and sending the second category vector and the target backbone network parameters of the target face recognition model to the server. Through the equivalent category vector, the category vector is not shared between the target clients, thereby ensuring the user's privacy information and improving the accuracy of user face recognition.
[0121] Optionally, the loss function of the target face recognition model is determined based on the cosine similarity between the feature vector of the sample face image and the first category vector of the target client, and the cosine similarity between the feature vector of the sample face image and the first category vector and the equivalent category vector matrix.
[0122] Optionally, the loss function is expressed by formula (1):
[0123]
[0124] Wherein, N represents the number of sample face images of the target client; represents the cosine similarity between the category vector of the i-th sample and the feature vector of the i-th sample, i represents the sample of the client, y i is the category shown by the i-th sample; s j,i Represents the category vector w j and the cosine similarity between the feature vector of the i-th sample, j represents the category vector of each client; C refers to the sum of the number of category vectors of each client and the number of equivalent category vectors.
[0125] Optionally, the face recognition model training device 600 based on federated learning further includes:
[0126] The third sending module is configured to send the first identifier of the target client to the server; the first identifier is used to represent the target client.
[0127] Figure 6 The second structural diagram of the face recognition model training device based on federated learning provided by the present invention is applied to the server, such as Figure 6As shown, the face recognition model training device 600 based on federated learning includes: a fusion module 601, a second sending module 602, a second receiving module 603 and an aggregation module 604; wherein,
[0128] A fusion module 601 is configured to perform fusion calculation on the category vectors of Cm candidate clients among the C candidate clients to obtain an equivalent category vector matrix; wherein the equivalent category vector matrix includes n equivalent category vectors;
[0129] A second sending module 602 is configured to send the first backbone network parameter and the equivalent category vector matrix to m candidate clients other than the Cm candidate clients among the C candidate clients respectively;
[0130] A second receiving module 603 is configured to receive the target backbone network parameters and the second category vector sent by each of the m candidate clients;
[0131] The aggregation module 604 is configured to aggregate the target backbone network parameters sent by each of the m candidate clients to obtain second backbone network parameters.
[0132] The present invention provides a face recognition model training device based on federated learning. The server calculates and fuses the category vectors of Cm candidate clients to obtain an equivalent category vector matrix. The server then sends the equivalent category vector matrix and first backbone network parameters to m candidate clients, respectively, for the m candidate clients to train the face recognition model. After training, the server receives the target backbone network parameters and second category vectors sent by each of the m candidate clients participating in the training. The server then aggregates the target backbone network parameters sent by each of the m candidate clients to obtain the second backbone network parameters. The equivalent category vectors prevent the m candidate clients from sharing the category vectors, thus protecting user privacy and improving the accuracy of user face recognition. Furthermore, after training, the equivalent category vectors do not need to be sent to the server, thus having no impact on upload bandwidth.
[0133] Optionally, the fusion module 604 is specifically configured to:
[0134] Randomly select n pairs of clients from the Cm candidate clients, perform fusion calculation on the category vectors of each pair of clients in the n pairs of clients, and obtain n equivalent category vectors.
[0135] Optionally, the fusion module 604 is specifically configured to:
[0136] The category vectors of each pair of clients in the n pairs of clients are respectively fused and calculated using formula (2) and formula (3) to obtain n equivalent category vectors;
[0137]
[0138]
[0139] Among them, the w i and w j Represents the category vector of each pair of clients; For the w i and w j The corresponding equivalent category vector.
[0140] Figure 7 A schematic diagram of the physical structure of a target client provided by the present invention, such as Figure 7 As shown, the target client 700 may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the face recognition model training method based on federated learning, which includes:
[0141] Receive an equivalent category vector matrix and first backbone network parameters sent by a server; the equivalent category vector matrix includes: n equivalent category vectors; the n equivalent category vectors are obtained by the server fusing the category vectors of Cm candidate clients; based on the first backbone network parameters, the sample face image of the target client, the first category vector of the target client and the equivalent category vector matrix, train a face recognition model to obtain a target face recognition model and a second category vector of the target client; send the second category vector and the target backbone network parameters of the target face recognition model to the server.
[0142] Optionally, the present invention further provides a server, which may include: a processor, a communications interface, a memory, and a communications bus, wherein the processor, the communications interface, and the memory communicate with each other via the communications bus. The processor may call logic instructions in the memory to execute a face recognition model training method based on federated learning, the method comprising:
[0143] Perform a fusion calculation on the category vectors of Cm candidate clients among the C candidate clients to obtain an equivalent category vector matrix; wherein the equivalent category vector matrix includes n equivalent category vectors; send first backbone network parameters and the equivalent category vector matrix to m candidate clients other than the Cm candidate clients among the C candidate clients respectively; receive the target backbone network parameters and the second category vector sent by each candidate client among the m candidate clients; aggregate the target backbone network parameters sent by each candidate client among the m candidate clients to obtain second backbone network parameters.
[0144] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0145] On the other hand, the present invention further provides a computer program product, comprising a computer program, which may be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is capable of executing the above-mentioned federated learning-based face recognition model training method applied to a target client, the method comprising:
[0146] Receive an equivalent category vector matrix and first backbone network parameters sent by a server; the equivalent category vector matrix includes: n equivalent category vectors; the n equivalent category vectors are obtained by the server fusing the category vectors of Cm candidate clients; based on the first backbone network parameters, the sample face image of the target client, the first category vector of the target client and the equivalent category vector matrix, train a face recognition model to obtain a target face recognition model and a second category vector of the target client; send the second category vector and the target backbone network parameters of the target face recognition model to the server.
[0147] On the other hand, the present invention further provides a computer program product, comprising a computer program, which may be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is capable of executing the above-mentioned federated learning-based face recognition model training method applied to a server, the method comprising:
[0148] Perform a fusion calculation on the category vectors of Cm candidate clients among the C candidate clients to obtain an equivalent category vector matrix; wherein the equivalent category vector matrix includes n equivalent category vectors; send first backbone network parameters and the equivalent category vector matrix to m candidate clients other than the Cm candidate clients among the C candidate clients respectively; receive the target backbone network parameters and the second category vector sent by each candidate client among the m candidate clients; aggregate the target backbone network parameters sent by each candidate client among the m candidate clients to obtain second backbone network parameters.
[0149] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the above-mentioned federated learning-based face recognition model training method applied to a target client, the method comprising:
[0150] Receive an equivalent category vector matrix and first backbone network parameters sent by a server; the equivalent category vector matrix includes: n equivalent category vectors; the n equivalent category vectors are obtained by the server fusing the category vectors of Cm candidate clients; based on the first backbone network parameters, the sample face image of the target client, the first category vector of the target client and the equivalent category vector matrix, train a face recognition model to obtain a target face recognition model and a second category vector of the target client; send the second category vector and the target backbone network parameters of the target face recognition model to the server.
[0151] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the above-mentioned federated learning-based face recognition model training method applied to a server, the method comprising:
[0152] Perform a fusion calculation on the category vectors of Cm candidate clients among the C candidate clients to obtain an equivalent category vector matrix; wherein the equivalent category vector matrix includes n equivalent category vectors; send first backbone network parameters and the equivalent category vector matrix to m candidate clients other than the Cm candidate clients among the C candidate clients respectively; receive the target backbone network parameters and the second category vector sent by each candidate client among the m candidate clients; aggregate the target backbone network parameters sent by each candidate client among the m candidate clients to obtain second backbone network parameters.
[0153] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.
[0154] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.
[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims
1. A face recognition model training method based on federated learning, characterized in that: Applied to a target client, the method includes: Receive an equivalent category vector matrix and a first backbone network parameter sent by a server; the equivalent category vector matrix includes: n equivalent category vectors; the server randomly selects n pairs of clients from Cm candidate clients among C candidate clients, and performs fusion calculation on the category vector of each pair of clients in the n pairs of clients using formula (2) and formula (3) to obtain n equivalent category vectors; Among them, the w i and w j Represents the category vector of each pair of clients; For the w i and w j The corresponding equivalent category vector; Training a face recognition model based on the first backbone network parameters, the sample face image of the target client, the first category vector of the target client, and the equivalent category vector matrix to obtain a target face recognition model and a second category vector of the target client; Sending the second category vector and the target backbone network parameters of the target face recognition model to the server.
2. The face recognition model training method based on federated learning according to claim 1, characterized in that: The loss function of the target face recognition model is determined based on the cosine similarity between the feature vector of the sample face image and the first category vector of the target client, and the cosine similarity between the feature vector of the sample face image and the first category vector and the equivalent category vector matrix.
3. The face recognition model training method based on federated learning according to claim 2, characterized in that: The loss function is expressed by formula (1): Wherein, N represents the number of sample face images of the target client; represents the cosine similarity between the category vector of the i-th sample and the feature vector of the i-th sample, i represents the sample of the client, y i is the category shown by the i-th sample; s j,i Represents the category vector w j and the cosine similarity between the feature vector of the i-th sample, j represents the category vector of each client; C refers to the sum of the number of category vectors of each client and the number of equivalent category vectors.
4. The face recognition model training method based on federated learning according to claim 1, characterized in that: Before receiving the equivalent category vector matrix and the first backbone network parameter sent by the server, the method further includes: Sending a first identifier of the target client to the server; the first identifier is used to represent the target client.
5. A face recognition model training method based on federated learning, characterized in that: Applied to a server, the method includes: Performing a fusion calculation on the category vectors of Cm candidate clients among the C candidate clients to obtain an equivalent category vector matrix; wherein the equivalent category vector matrix includes n equivalent category vectors; Sending the first backbone network parameter and the equivalent category vector matrix to m candidate clients other than the Cm candidate clients among the C candidate clients respectively; Receiving a target backbone network parameter and a second category vector sent by each of the m candidate clients; aggregating the target backbone network parameters sent by each of the m candidate clients to obtain second backbone network parameters; The fusion calculation of the category vectors of Cm candidate clients among the C candidate clients to obtain an equivalent category vector matrix includes: Randomly select n pairs of clients from the Cm candidate clients, and perform fusion calculation on the category vectors of each pair of clients in the n pairs of clients using formula (2) and formula (3) to obtain n equivalent category vectors; Among them, the w i and w j Represents the category vector of each pair of clients; For the w i and w j The corresponding equivalent category vector.
6. A face recognition model training device based on federated learning, characterized in that: Applied to a target client, the device comprises: A first receiving module is configured to receive an equivalent category vector matrix and a first backbone network parameter sent by a server; the equivalent category vector matrix includes: n equivalent category vectors; the server randomly selects n pairs of clients from Cm candidate clients among C candidate clients, and performs fusion calculation on the category vectors of each pair of clients in the n pairs of clients using formula (2) and formula (3) to obtain n equivalent category vectors; Among them, the w i and w j Represents the category vector of each pair of clients; For the w i and w j The corresponding equivalent category vector; a training module, configured to train a face recognition model based on the first backbone network parameters, a sample face image of the target client, the first category vector of the target client, and the equivalent category vector matrix, to obtain a target face recognition model and a second category vector of the target client; The first sending module is used to send the second category vector and the target backbone network parameters of the target face recognition model to the server.
7. A face recognition model training device based on federated learning, characterized in that: Applied to a server, the device includes: A fusion module, configured to perform fusion calculation on the category vectors of Cm candidate clients among the C candidate clients to obtain an equivalent category vector matrix; wherein the equivalent category vector matrix includes n equivalent category vectors; A second sending module is configured to send the first backbone network parameter and the equivalent category vector matrix to m candidate clients other than the Cm candidate clients among the C candidate clients respectively; A second receiving module is configured to receive a target backbone network parameter and a second category vector sent by each of the m candidate clients; an aggregation module, configured to aggregate the target backbone network parameters sent by each of the m candidate clients to obtain a second backbone network parameter; The fusion module is specifically used to: Randomly select n pairs of clients from the Cm candidate clients, and perform fusion calculation on the category vectors of each pair of clients in the n pairs of clients using formula (2) and formula (3) to obtain n equivalent category vectors; Among them, the w i and w j Represents the category vector of each pair of clients; For the w i and w j The corresponding equivalent category vector.
8. A target client comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the face recognition model training method based on federated learning are implemented as described in any one of claims 1 to 4.
9. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the face recognition model training method based on federated learning as claimed in claim 5 are implemented.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the face recognition model training method based on federated learning are implemented as described in any one of claims 1 to 4, or the steps of the face recognition model training method based on federated learning as described in claim 5.
11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the face recognition model training method based on federated learning are implemented as described in any one of claims 1 to 4, or the steps of the face recognition model training method based on federated learning as described in claim 5.