Identity recognition method and device and identity recognition model training method and device
By using the general model and local model to extract features in federated learning and training the local model based on personalized differentiation loss function, the problem that the general model cannot adapt to local features is solved, reducing the model time-consuming and improving the recognition performance.
Patent Information
- Application Number
- CN202410991190.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-07-23
AI Technical Summary
In the face recognition, the existing federated learning method is unable to effectively adapt to local features, resulting in increased model time-consuming and reduced localized recognition performance.
By sampling private data sets on the client, extract features using general models and local models, and train local models based on general features, local features and preset personalized distinction loss functions, optimize local models to distinguish local features, and generate pre-training identification model.
This method reduces the time-consuming model and improves the recognition performance of the local model, making local features more distinctive in the local dataset, and enhancing the model's adaptability to the data distribution of different client sides.
Smart Images

Figure CN119091480A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine learning technology, and in particular to a method and device for identity recognition and model training thereof. Background Art
[0002] Federated Learning (FL) is a distributed machine learning method that allows multiple clients (such as mobile devices, browsers, edge devices, etc.) to jointly train a shared machine learning model with the server while maintaining data privacy and localization. Federated learning has been applied to face recognition tasks, but this method is to jointly train a universal model. Since face recognition data may contain local features (such as camera type, race, etc.), the universal model cannot adapt well to local recognition scenarios. Therefore, how to provide each client with a local personalized model while obtaining a universal model is a great demand.
[0003] In the related technology, the method adopted is to use a preset conversion layer to convert the general features into local features after the general model extracts features. The preset conversion layer requires additional calculation of the conversion, which increases the time consumption of the model. At the same time, the extracted general features themselves are not for local services, so some local features that are crucial for personalized recognition may have been lost in the feature extraction process, thereby reducing the localized recognition performance of the model. Summary of the invention
[0004] The embodiments of the present application provide a method and device for identity recognition and model training thereof. In order to have a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. This summary is not a general review, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its only purpose is to present some concepts in a simple form as a preface to the detailed description that follows.
[0005] In a first aspect, an embodiment of the present application provides a recognition model training method, which is applied to a client, wherein the client includes a private data set, a general model for federated learning, and a local model, wherein the general model is a model shared by all clients, and the local model is trained for a local service, and the method includes: Sampling the private data set to obtain training data; Extract features from the training data using the general model and the local model to obtain general features belonging to the general model and local features belonging to the local model; The local model is trained according to the common features, the local features and the preset personalized distinguishing loss function, and the preset personalized distinguishing loss function is used to optimize the local model so that the local model can distinguish the local features in the private data set; When the private dataset sampling is completed, a pre-trained recognition model is generated.
[0006] Optionally, the general model includes a first classifier layer, and the training data includes face images and their corresponding identity labels; Before distinguishing the loss function based on common features, local features, and preset personalization, it also includes: Input the common features into the first classifier layer to obtain a first classification result; Calculate a first general face recognition loss based on the first classification result and the identity label in combination with a preset face recognition loss function; Based on the first universal face recognition loss, model parameters of the universal model are updated.
[0007] Optionally, the local model includes a batch normalization layer and a second classifier layer, and the batch normalization layer is the same as the normalization layer of the feature extractor included in the general model; The method also includes: Input the local features into the second classifier layer to obtain a second classification result; Calculate a second general face recognition loss based on the second classification result and the identity label in combination with a preset face recognition loss function; Based on the second general face recognition loss, update the personalized parameters of the batch normalization layer.
[0008] Optionally, a local model is trained based on common features, local features, and a preset personalized differentiation loss function, including: Calculate the cosine similarity between local features and universal features; Calculate target similarity between common features; Calculate the personalized distinction loss value according to the preset personalized distinction loss function, cosine similarity and target similarity; Calculate the face recognition loss value according to the preset face recognition loss function, common features and local features; Calculate the sum of the personalized distinction loss value and the face recognition loss value to obtain the final loss value of the model parameters for optimizing the local model; Based on the final loss value, optimize the model parameters of the local model.
[0009] Optionally, according to a preset personalized distinction loss function, cosine similarity and target similarity, a personalized distinction loss value is calculated, including: Construct a mask matrix based on cosine similarity and target similarity; Divide the training data into positive sample pairs representing the face images of the same identity and negative sample pairs representing the face images of different identities; For each positive sample pair and negative sample pair indicated by each element in the mask matrix, compare the cosine similarity associated with each element with the target similarity; Based on the comparison results, the positive sample pairs and negative sample pairs are penalized by presetting a personalized discrimination loss function to encourage the local model to show greater similarity between positive sample pairs and smaller similarity between negative sample pairs, thereby obtaining a personalized discrimination loss value.
[0010] Optionally, based on the comparison result, the positive sample pairs and the negative sample pairs are penalized by a preset personalized distinguishing loss function, including: When the comparison result indicates that the cosine similarity is less than the target similarity, the positive sample pair is penalized by a preset personalized discrimination loss function; or, When the comparison result indicates that the cosine similarity is greater than the difference between the target similarity and the preset boundary value, the negative sample pair is penalized by a preset personalized discrimination loss function.
[0011] Optionally, the preset personalized discrimination loss function is: in, To personalize the loss value, is the batch normalization layer of the local model, For a single communication round, For local features, For general features, is the cosine similarity between local features and universal features, is the target similarity between common features, is the mask value. When the universal feature indicated by each element in the mask matrix is the same as the local feature, the mask value of each element is set to 1, or when the universal feature indicated by each element in the mask matrix is different from the local feature, the mask value of each element is set to 0. is the preset boundary value.
[0012] In a second aspect, an embodiment of the present application provides an identity recognition method, the method comprising: receiving a face recognition request, the face recognition request carrying a user's face image; Inputting the face image into a pre-trained recognition model and outputting the identity features of the user; the pre-trained recognition model is trained by the above-mentioned recognition model training method; The user's identity features are compared with the identity features registered by the user to obtain the user's comparison result.
[0013] In a third aspect, an embodiment of the present application provides a recognition model training device, the device including a private data set, a general model deployed for federated learning, and a local model, the general model is a model shared by all clients, and the local model is trained for a local service, and the device includes: The sampling module is used to sample the private data set to obtain training data; A feature extraction module is used to extract features from training data using a general model and a local model to obtain general features belonging to the general model and local features belonging to the local model; A training module, used for training a local model according to common features, local features and a preset personalized distinguishing loss function, wherein the preset personalized distinguishing loss function is used for optimizing the local model so that the local model can distinguish the local features in the private data set; The generation module is used to generate a pre-trained recognition model when the private data set sampling is completed.
[0014] In a fourth aspect, an embodiment of the present application provides an identity recognition device, the device comprising: A receiving module, used to receive a face recognition request, the face recognition request carries a user's face image; An output module, used to input a face image into a pre-trained recognition model and output the identity features of the user; the pre-trained recognition model is trained by the above-mentioned recognition model training method; The comparison module is used to compare the user's identity characteristics with the user's registered identity characteristics to obtain the user's comparison result.
[0015] The technical solution provided by the embodiments of the present application may have the following beneficial effects: In the embodiments of the present application, on the one hand, the present application trains a local model based on common features, local features and a preset personalized discrimination loss function. This process does not require additional calculation of the conversion, thereby reducing the time consumption of the model. On the other hand, the preset personalized discrimination loss function provided by the present application is used to optimize the local model, so that the local model can distinguish the local features in the private data set, ensuring that the local features are more discriminative than the common features in the local data set, thereby improving the recognition performance of the local model.
[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0018] Figure 1It is a flowchart of a recognition model training method provided in an embodiment of the present application; Figure 2 This is a schematic diagram of a model training architecture under federated learning provided in an embodiment of the present application; Figure 3 It is a flowchart of an identity recognition method provided in an embodiment of the present application; Figure 4 It is a structural schematic diagram of a recognition model training device provided in an embodiment of the present application; Figure 5 is a structural schematic diagram of an identity recognition device provided in an embodiment of the present application; Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0019] The following description and the accompanying drawings sufficiently illustrate specific embodiments of the present application to enable those skilled in the art to practice them.
[0020] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.
[0021] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.
[0022] In the description of the present application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.
[0023] It should be noted that in this application, when collecting facial images, the user will be clearly informed of the purpose, and the collection will be implemented only after obtaining the user's consent, and the collected facial images are within the necessary scope. For example, there is a sampling module in this application. When executing the sampling module, the user will be clearly informed of the purpose of collecting facial images, that is, the user will be informed that the sampling module will collect facial images, and the collected facial data will be used for model training only after obtaining the user's consent, and the facial images collected by the sampling module are within the necessary scope.
[0024] The present application provides a method and device for identity recognition and model training to solve the problems existing in the above-mentioned related technical issues. In the embodiments of the present application, on the one hand, the present application trains a local model based on common features, local features and a preset personalized distinguishing loss function. This process does not require additional calculation of the conversion, which reduces the time consumption of the model; on the other hand, the preset personalized distinguishing loss function provided by the present application is used to optimize the local model so that the local model can distinguish the local features in the private data set, ensuring that the local features are more distinguishable than the common features in the local data set, thereby improving the recognition performance of the local model. The following is a detailed description using an exemplary embodiment.
[0025] The present application provides an identity recognition and model training method and device to solve the problems existing in the above-mentioned related technical issues. Figure 1 -Attached Figure 3 , the recognition model training method provided in the embodiment of the present application is introduced in detail. The method can be implemented by a computer program and can be run on a recognition model training device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application.
[0026] See also Figure 1 , provides a flow chart of a recognition model training method according to an embodiment of the present application, which is applied to a client, the client including a private data set, a general model deployed for federated learning, and a local model, the general model is a model shared by all clients, and the local model is trained for local services. Figure 1 As shown, the method of the embodiment of the present application may include the following steps: S101, sampling the private data set to obtain training data; Among them, the private dataset can be understood as assuming that there is a dataset containing N samples D , where each sample Represents a face image and its corresponding identity label, with a total of n different identities. A deep face recognition model uses a feature extractor Ø to generate a feature vector , and through a weighted The classifier layer is used for classification. A softmax-based loss function is usually used to optimize these parameters, as follows: ; Among them, the function and It varies in different methods. For example, CosFace uses and , while ArcFace uses and .here is a hyperparameter, is a vector and The angle between.
[0027] In the federated learning environment of this application, it is assumed that there is a server and clients, each client There is a private dataset , on each client In the deployment of a general model and client-specific local model parameters ,in, Represents the batch normalization layer, whose structure is similar to the feature extractor The normalization layer in is the same, and all clients Have the same network architecture.
[0028] Compared with traditional FedAvg, this application has two main differences: (1) The classifier layers are kept locally because they are dispensable for reasoning and directly involve personal privacy. (2) Each client $c$ has a local model ,in Represents a function that combines batch normalization parameters and The other parameters of the feature extractor are used to generate the feature extractor.
[0029] In one embodiment, firstly, the private data sets owned by each client are determined. These data sets contain locally collected facial images and their corresponding identity tags. Then, before sampling, necessary preprocessing is performed on the data sets, including image cropping, scaling, normalization, etc., to ensure data consistency and model input requirements. Secondly, according to the training objectives and the characteristics of the data sets, a suitable sampling strategy is selected. The adopted strategies may include random sampling, stratified sampling, oversampling or undersampling, etc. Finally, a certain number of samples are randomly selected from the private data sets to form a training batch of training data.
[0030] For example, given a private dataset , each iteration draws a small batch of samples from .
[0031] S102, extracting features from the training data using the general model and the local model to obtain general features belonging to the general model and local features belonging to the local model; Among them, the general model is a model shared by all clients in federated learning. It is initialized by the central server and distributed to each client. The purpose is to capture the common features across clients and provide basic recognition capabilities for all clients. During the federated learning process, the central server will collect model updates from all clients and aggregate them to improve the general model. The local model is a model optimized for a specific client. It contains some client-specific parameters, such as the parameters of the batch normalization layer. The purpose is to improve the model's adaptability to specific client data sets and achieve more accurate face recognition.
[0032] In some embodiments, the general model and the local model are first initialized, and then the training data are respectively input into the initialized general model and the local model, and feature extraction is performed through the feature extractor of the model to obtain general features belonging to the general model and local features belonging to the local model.
[0033] For example, for a sample , and its corresponding features are extracted as ,in Generate a combined and Weighted feature extractor.
[0034] The general model includes a first classifier layer, and the training data includes face images and their corresponding identity labels.
[0035] In an embodiment of the present application, the process of updating the model parameters of the general model includes: inputting the general features into the first classifier layer to obtain a first classification result; calculating the first general face recognition loss based on the first classification result and the identity label in combination with a preset face recognition loss function; and updating the model parameters of the general model based on the first general face recognition loss.
[0036] The local model includes a batch normalization layer and a second classifier layer, and the batch normalization layer is the same as the normalization layer of the feature extractor included in the general model.
[0037] In an embodiment of the present application, the process of updating the model parameters of the local model includes: inputting local features into a second classifier layer to obtain a second classification result; calculating a second universal face recognition loss based on the second classification result and the identity label in combination with a preset face recognition loss function; and updating the personalized parameters of the batch normalization layer based on the second universal face recognition loss.
[0038] The default face recognition loss function is: ;in, The second general face recognition loss, For the Batch normalization layer with communication rounds, For the The classifier layer under communication rounds, is the number of training samples, and are the first and second functions respectively, is a natural number, is the ith adjacent eigenvector, For feature marking.
[0039] S103, training a local model according to the common features, the local features, and a preset personalized distinguishing loss function, wherein the preset personalized distinguishing loss function is used to optimize the local model so that the local model can distinguish the local features in the private data set; In the embodiment of the present application, the specific process of training the local model according to the common features, local features and the preset personalized distinguishing loss function includes: calculating the cosine similarity between the local features and the common features; calculating the target similarity between the common features; calculating the personalized distinguishing loss value according to the preset personalized distinguishing loss function, the cosine similarity and the target similarity; calculating the face recognition loss value according to the preset face recognition loss function, the common features and the local features; calculating the sum of the personalized distinguishing loss value and the face recognition loss value to obtain the final loss value of the model parameters of the optimized local model; optimizing the model parameters of the local model based on the final loss value. Optimizing the parameters of the local model The final training loss is: in, is the final training loss, is the face recognition loss value, To personalize the loss value, is a hyperparameter, and this loss only affects the parameter , without affecting the general model parameters.
[0040] For example, the cosine similarity between features: , similarly, the target similarity between common features is calculated ,get .
[0041] Specifically, according to the preset personalized discrimination loss function, cosine similarity and target similarity, the process of calculating the personalized discrimination loss value includes: constructing a mask matrix according to the cosine similarity and the target similarity; dividing the positive sample pairs representing the face images of the same identity and the negative sample pairs representing the face images of different identities from the training data; for the positive sample pairs and negative sample pairs indicated by each element in the mask matrix, comparing the size between the cosine similarity associated with each element and the target similarity; based on the comparison result, penalizing the positive sample pairs and the negative sample pairs by means of the preset personalized discrimination loss function to encourage the local model to show greater similarity between the positive sample pairs and smaller similarity between the negative sample pairs, so as to obtain the personalized discrimination loss value.
[0042] Specifically, based on the comparison result, the process of penalizing the positive sample pairs and the negative sample pairs by using the preset personalized discrimination loss function includes: when the comparison result indicates that the cosine similarity is less than the target similarity, the positive sample pairs are penalized by using the preset personalized discrimination loss function; or, when the comparison result indicates that the cosine similarity is greater than the difference between the target similarity and the preset boundary value, the negative sample pairs are penalized by using the preset personalized discrimination loss function.
[0043] The preset personalized differentiation loss function is: in, To personalize the loss value, is the batch normalization layer of the local model, For a single communication round, For local features, For general features, is the cosine similarity between local features and universal features, is the target similarity between common features, is the mask value. When the universal feature indicated by each element in the mask matrix is the same as the local feature, the mask value of each element is set to 1, or when the universal feature indicated by each element in the mask matrix is different from the local feature, the mask value of each element is set to 0. is the preset boundary value.
[0044] For example, the mask matrix ,in express ,otherwise .
[0045] S104, when the private data set sampling is completed, generate a pre-trained recognition model.
[0046] In an embodiment of the present application, after the training is completed, it can be determined whether the sampling of the private data set has been completed. If the sampling of the private data set has been completed, a pre-trained recognition model is generated; or if the sampling of the private data set has not been completed, the step of sampling the private data set is continued.
[0047] For example Figure 2 As shown, Figure 2 This is a schematic diagram of a model training architecture under federated learning provided by this application. The architecture consists of a server and a client. The server sends training auxiliary parameters to the client, and the client uploads the trained general model to the server for aggregation. In addition, the client also trains a local personalized local model by sampling in the data set saved by the client, inputting the sampled training data into the general model and the local model, obtaining the general features belonging to the general model and the local features belonging to the local model, and then optimizing the parameters of the local model in combination with the preset personalized discrimination loss function (i.e., the discrimination loss function).
[0048] In an embodiment of the present application, by presetting a personalized discriminative loss function (PDLoss), it can be ensured that local features are more discriminative than universal features in the local data set. This optimization not only improves the recognition accuracy of the model, but also enhances the model's adaptability to different client data distributions, and effectively addresses the problem of data heterogeneity.
[0049] In the embodiment of the present application, the design of independently updating the batch normalization layer parameters in the model training architecture of the present application enables the model to better adapt to the characteristics of local data, improving the training efficiency and model performance. This mechanism enables the global model and the local model to be more coordinated during the training process, thereby improving the convergence speed and stability of the overall model.
[0050] In the embodiments of the present application, on the one hand, the present application trains a local model based on common features, local features and a preset personalized discrimination loss function. This process does not require additional calculation of the conversion, thereby reducing the time consumption of the model. On the other hand, the preset personalized discrimination loss function provided by the present application is used to optimize the local model, so that the local model can distinguish the local features in the private data set, ensuring that the local features are more discriminative than the common features in the local data set, thereby improving the recognition performance of the local model.
[0051] See also Figure 3 , which is a flow chart of an identity recognition method provided in an embodiment of the present application. Figure 3 As shown, the method of the embodiment of the present application may include the following steps: S201, receiving a face recognition request, the face recognition request carrying a face image of a user; In some embodiments, a facial recognition request is received, the facial recognition request is parsed, and a facial image of the user is extracted from the parsed data.
[0052] S202, inputting the face image into a pre-trained recognition model, and outputting the identity features of the user; the pre-trained recognition model is trained by the above-mentioned recognition model training method; The pre-trained recognition model can be a local model or a universal model.
[0053] In some embodiments, when the current recognition task needs to be performed on the server side, the facial image is processed through the general model of the server side to obtain the user's identity features.
[0054] In other embodiments, when the current recognition task needs to be performed on the client, the facial image is processed by a local model of the client to obtain the user's identity features.
[0055] In an embodiment of the present application, a pre-trained recognition model can be generated according to the following steps, including: sampling a private data set to obtain training data; extracting features from the training data through a general model and a local model to obtain general features belonging to the general model and local features belonging to the local model; training a local model based on the general features, local features and a preset personalized distinction loss function, wherein the preset personalized distinction loss function is used to optimize the local model so that the local model can distinguish the local features in the private data set; when the sampling of the private data set is completed, a pre-trained recognition model is generated.
[0056] S203, comparing the identity characteristics of the user with the identity characteristics registered by the user to obtain a comparison result of the user.
[0057] In an embodiment of the present application, during the user registration stage, a face recognition model is used to extract the identity features of the face image submitted by the user when registering, and the extracted identity features are securely stored in a database and associated with the user's unique identifier (such as a user ID). The real identity features of the user registration are obtained through the user ID, and the user's identity features are compared with the identity features registered by the user to calculate a similarity score between the two feature vectors, which represents the degree of similarity between them. A similarity threshold is set to determine whether the comparison result indicates that the two features belong to the same user. If the similarity score exceeds the threshold, the comparison is determined to be successful and the user's identity is confirmed; if it is lower than the threshold, the comparison is determined to have failed, and the comparison result is fed back to the user or system in an appropriate manner, such as by displaying it through a user interface.
[0058] In the embodiments of the present application, on the one hand, the present application trains a local model based on common features, local features and a preset personalized discrimination loss function. This process does not require additional calculation of the conversion, thereby reducing the time consumption of the model. On the other hand, the preset personalized discrimination loss function provided by the present application is used to optimize the local model, so that the local model can distinguish the local features in the private data set, ensuring that the local features are more discriminative than the common features in the local data set, thereby improving the recognition performance of the local model.
[0059] The following are device embodiments of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.
[0060] See also Figure 4 , which shows a schematic diagram of the structure of a recognition model training device provided by an exemplary embodiment of the present application. The recognition model training device can be implemented as all or part of an electronic device through software, hardware, or a combination of both. The device includes a private data set, a general model deployed for federated learning, and a local model. The general model is a model shared by all clients, and the local model is trained for local services. The device 1 includes a sampling module 10, a feature extraction module 20, a training module 30, and a generation module 40.
[0061] A sampling module 10, used to sample the private data set to obtain training data; A feature extraction module 20, used to extract features from training data using a universal model and a local model to obtain universal features belonging to the universal model and local features belonging to the local model; A training module 30, used for training a local model according to common features, local features and a preset personalized distinguishing loss function, wherein the preset personalized distinguishing loss function is used to optimize the local model so that the local model can distinguish the local features in the private data set; The generation module 40 is used to generate a pre-trained recognition model when the private data set sampling is completed.
[0062] Optionally, the device further comprises: A first input module, used for inputting the common features into a first classifier layer to obtain a first classification result; A first loss calculation module, used to calculate a first general face recognition loss according to the first classification result and the identity label in combination with a preset face recognition loss function; The first updating module is used to update the model parameters of the general model based on the first general face recognition loss.
[0063] Optionally, the device further comprises: A second input module, used for inputting the local features into a second classifier layer to obtain a second classification result; A second loss calculation module, used to calculate a second general face recognition loss based on the second classification result and the identity label in combination with a preset face recognition loss function; The second updating module is used to update the personalized parameters of the batch normalization layer based on the second general face recognition loss.
[0064] It should be noted that the recognition model training device provided in the above embodiment only uses the division of the above functional modules as an example when executing the recognition model training method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the recognition model training device provided in the above embodiment belongs to the same concept as the recognition model training method embodiment. The implementation process is detailed in the method embodiment and will not be repeated here.
[0065] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0066] In the embodiments of the present application, on the one hand, the present application trains a local model based on common features, local features and a preset personalized discrimination loss function. This process does not require additional calculation of the conversion, thereby reducing the time consumption of the model. On the other hand, the preset personalized discrimination loss function provided by the present application is used to optimize the local model, so that the local model can distinguish the local features in the private data set, ensuring that the local features are more discriminative than the common features in the local data set, thereby improving the recognition performance of the local model.
[0067] See also Figure 5 , which shows a schematic diagram of the structure of an identity recognition device provided by an exemplary embodiment of the present application. The recognition model training device can be implemented as all or part of an electronic device through software, hardware or a combination of both. The device 2 includes a receiving module 10, an output module 20, and a comparison module 30.
[0068] A receiving module 10 is used to receive a face recognition request, where the face recognition request carries a face image of a user; An output module 20, used to input a face image into a pre-trained recognition model and output a user's identity features; the pre-trained recognition model is obtained by training the recognition model training method according to any one of claims 1 to 7; The comparison module 30 is used to compare the identity characteristics of the user with the identity characteristics registered by the user to obtain the comparison result of the user.
[0069] It should be noted that the recognition model training device provided in the above embodiment only uses the division of the above functional modules as an example when executing the recognition model training method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the recognition model training device provided in the above embodiment belongs to the same concept as the recognition model training method embodiment. The implementation process is detailed in the method embodiment and will not be repeated here.
[0070] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0071] In the embodiments of the present application, on the one hand, the present application trains a local model based on common features, local features and a preset personalized discrimination loss function. This process does not require additional calculation of the conversion, thereby reducing the time consumption of the model. On the other hand, the preset personalized discrimination loss function provided by the present application is used to optimize the local model, so that the local model can distinguish the local features in the private data set, ensuring that the local features are more discriminative than the common features in the local data set, thereby improving the recognition performance of the local model.
[0072] The present application also provides a computer-readable medium having program instructions stored thereon, which, when executed by a processor, implement the recognition model training methods provided by the above-mentioned various method embodiments.
[0073] The present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the recognition model training methods of the above-mentioned various method embodiments.
[0074] See also Figure 6 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device 1000 may include: at least one processor 1001 , at least one network interface 1004 , a user interface 1003 , a memory 1005 , and at least one communication bus 1002 .
[0075] The communication bus 1002 is used to realize the connection and communication between these components.
[0076] The user interface 1003 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.
[0077] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).
[0078] Among them, the processor 1001 may include one or more processing cores. The processor 1001 uses various interfaces and lines to connect various parts of the entire electronic device 1000, and executes various functions and processes data of the electronic device 1000 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1005, and calling data stored in the memory 1005. Optionally, the processor 1001 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 1001 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 1001, and it can be implemented separately through a chip.
[0079] Among them, the memory 1005 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 1005 may also be optionally at least one storage system located away from the aforementioned processor 1001. As Figure 6 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a recognition model training application.
[0080] exist Figure 6In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 1001 can be used to call the recognition model training application stored in the memory 1005, and specifically perform the following operations: Sampling the private data set to obtain training data; Extract features from the training data using the general model and the local model to obtain general features belonging to the general model and local features belonging to the local model; The local model is trained according to the common features, the local features and the preset personalized distinguishing loss function, and the preset personalized distinguishing loss function is used to optimize the local model so that the local model can distinguish the local features in the private data set; When the private dataset sampling is completed, a pre-trained recognition model is generated.
[0081] In one embodiment, the processor 1001 further performs the following operations: Input the common features into the first classifier layer to obtain a first classification result; Calculate a first general face recognition loss based on the first classification result and the identity label in combination with a preset face recognition loss function; Based on the first universal face recognition loss, model parameters of the universal model are updated.
[0082] In one embodiment, the processor 1001 further performs the following operations: Input the local features into the second classifier layer to obtain a second classification result; Calculate a second general face recognition loss based on the second classification result and the identity label in combination with a preset face recognition loss function; Based on the second general face recognition loss, update the personalized parameters of the batch normalization layer.
[0083] In one embodiment, when the processor 1001 trains the local model according to the common features, the local features, and the preset personalized differentiation loss function, the processor 1001 specifically performs the following operations: Calculate the cosine similarity between local features and universal features; Calculate target similarity between common features; Calculate the personalized distinction loss value according to the preset personalized distinction loss function, cosine similarity and target similarity; Calculate the face recognition loss value according to the preset face recognition loss function, common features and local features; Calculate the sum of the personalized distinction loss value and the face recognition loss value to obtain the final loss value of the model parameters for optimizing the local model; Based on the final loss value, optimize the model parameters of the local model.
[0084] In one embodiment, when the processor 1001 calculates the personalized distinction loss value according to the preset personalized distinction loss function, the cosine similarity and the target similarity, the processor 1001 specifically performs the following operations: Construct a mask matrix based on cosine similarity and target similarity; Divide the training data into positive sample pairs representing the face images of the same identity and negative sample pairs representing the face images of different identities; For each positive sample pair and negative sample pair indicated by each element in the mask matrix, compare the cosine similarity associated with each element with the target similarity; Based on the comparison results, the positive sample pairs and negative sample pairs are penalized by presetting a personalized discrimination loss function to encourage the local model to show greater similarity between positive sample pairs and smaller similarity between negative sample pairs, thereby obtaining a personalized discrimination loss value.
[0085] In one embodiment, when the processor 1001 penalizes the positive sample pairs and the negative sample pairs based on the comparison result by using a preset personalized discrimination loss function, the processor 1001 specifically performs the following operations: When the comparison result indicates that the cosine similarity is less than the target similarity, the positive sample pair is penalized by a preset personalized discrimination loss function; or, When the comparison result indicates that the cosine similarity is greater than the difference between the target similarity and the preset boundary value, the negative sample pair is penalized by a preset personalized discrimination loss function.
[0086] In the embodiments of the present application, on the one hand, the present application trains a local model based on common features, local features and a preset personalized discrimination loss function. This process does not require additional calculation of the conversion, thereby reducing the time consumption of the model. On the other hand, the preset personalized discrimination loss function provided by the present application is used to optimize the local model, so that the local model can distinguish the local features in the private data set, ensuring that the local features are more discriminative than the common features in the local data set, thereby improving the recognition performance of the local model.
[0087] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the recognition model training program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.
[0088] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.
Claims
1. A recognition model training method, characterized in that: Applied to a client, the client includes a private dataset, a general model deployed for federated learning, and a local model, the general model is a model shared by all clients, and the local model is trained for a local service, the method includes: Sampling the private data set to obtain training data; Performing feature extraction on the training data through the universal model and the local model to obtain universal features belonging to the universal model and local features belonging to the local model; Training the local model according to the common features, the local features and a preset personalized distinction loss function, wherein the preset personalized distinction loss function is used to optimize the local model so that the local model can distinguish the local features in the private data set; When the sampling of the private data set is completed, a pre-trained recognition model is generated.
2. The method according to claim 1, characterized in that The general model includes a first classifier layer, and the training data includes face images and their corresponding identity tags; Before distinguishing the loss function according to the common features, the local features and the preset personalized features, the method further includes: Inputting the common features into the first classifier layer to obtain a first classification result; Calculate a first general face recognition loss based on the first classification result and the identity label in combination with a preset face recognition loss function; Based on the first universal face recognition loss, model parameters of the universal model are updated.
3. The method according to claim 2, characterized in that The local model includes a batch normalization layer and a second classifier layer, wherein the batch normalization layer is the same as the normalization layer of the feature extractor included in the general model; The method further comprises: Inputting the local features into the second classifier layer to obtain a second classification result; Calculate a second general face recognition loss based on the second classification result and the identity label in combination with a preset face recognition loss function; Based on the second universal face recognition loss, updating the personalized parameters of the batch normalization layer.
4. The method according to claim 1, characterized in that: The training of the local model according to the universal features, the local features and a preset personalized distinguishing loss function includes: Calculating the cosine similarity between the local feature and the universal feature; Calculating target similarity between the common features; Calculate a personalized distinction loss value according to a preset personalized distinction loss function, the cosine similarity and the target similarity; Calculating a face recognition loss value according to a preset face recognition loss function, the universal features and the local features; Calculating the sum of the personalized distinction loss value and the face recognition loss value to obtain a final loss value for optimizing the model parameters of the local model; Based on the final loss value, model parameters of the local model are optimized.
5. The method according to claim 4, characterized in that The calculating the personalized distinction loss value according to the preset personalized distinction loss function, the cosine similarity and the target similarity includes: Constructing a mask matrix according to the cosine similarity and the target similarity; Dividing from the training data positive sample pairs of face images representing the same identity and negative sample pairs of face images representing different identities; For the positive sample pair and the negative sample pair indicated by each element in the mask matrix, compare the cosine similarity associated with each element with the target similarity; Based on the comparison result, the positive sample pairs and the negative sample pairs are penalized by a preset personalized distinction loss function to encourage the local model to show greater similarity between the positive sample pairs and less similarity between the negative sample pairs, thereby obtaining a personalized distinction loss value.
6. The method according to claim 5, characterized in that Based on the comparison result, the positive sample pair and the negative sample pair are penalized by a preset personalized distinction loss function, including: When the comparison result indicates that the cosine similarity is less than the target similarity, the positive sample pair is penalized by a preset personalized distinction loss function; or, When the comparison result indicates that the cosine similarity is greater than the difference between the target similarity and a preset boundary value, the negative sample pair is penalized by a preset personalized distinction loss function.
7. The method according to claim 5, characterized in that The preset personalized distinction loss function is: in, To personalize the loss value, is the batch normalization layer of the local model, For a single communication round, For local features, For general features, is the cosine similarity between the local feature and the universal feature, is the target similarity between common features, is a mask value, when the universal feature indicated by each element in the mask matrix is the same as the local feature, the mask value of each element is set to 1, or when the universal feature indicated by each element in the mask matrix is different from the local feature, the mask value of each element is set to 0, is the preset boundary value.
8. An identity recognition method, characterized in that: The method comprises: receiving a face recognition request, wherein the face recognition request carries a face image of a user; Inputting the face image into a pre-trained recognition model to output the identity features of the user; the pre-trained recognition model is trained by the recognition model training method according to any one of claims 1 to 7; The identity feature of the user is compared with the identity feature registered by the user to obtain a comparison result of the user.
9. A recognition model training device, characterized in that: The device includes a private data set, a general model deployed for federated learning, and a local model, wherein the general model is a model shared by all clients, and the local model is trained for a local service, and the device includes: A sampling module, used for sampling the private data set to obtain training data; A feature extraction module, used to extract features from the training data using the general model and the local model to obtain general features belonging to the general model and local features belonging to the local model; A training module, used for training the local model according to the common features, the local features and a preset personalized distinguishing loss function, wherein the preset personalized distinguishing loss function is used for optimizing the local model so that the local model can distinguish the local features in the private data set; A generation module is used to generate a pre-trained recognition model when the sampling of the private data set is completed.
10. An identity recognition device, characterized in that: The device comprises: A receiving module, used to receive a face recognition request, wherein the face recognition request carries a face image of a user; An output module, used to input the face image into a pre-trained recognition model and output the identity features of the user; the pre-trained recognition model is trained by the recognition model training method according to any one of claims 1 to 7; The comparison module is used to compare the identity characteristics of the user with the identity characteristics registered by the user to obtain the comparison result of the user.
Citation Information
Patent Citations
Face recognition method and device based on knowledge federation, equipment, medium and system
CN111046857A
Face recognition method based on secret sharing and federated learning
CN115830679A
Federal learning system based on data-knowledge-free distillation execution model aggregation
CN116992957A
