Face attribute recognition method, client, device and medium based on federated learning

Abstract

The present invention relates to the field of financial technology, and discloses a face attribute recognition method, client, device and medium based on federated learning. The face attribute recognition method based on federated learning is applied to a client, and the face attribute recognition method includes: the client obtains a face image; the face attribute recognition model in the client recognizes the face image to obtain face attributes, and the face attribute recognition model is obtained by longitudinal federated learning based on face features provided by different clients; and the face attributes are output. The present invention overcomes the defect in the prior art that the client uses its own face image to train the face attribute recognition model, resulting in a low recognition accuracy rate of the client's face attributes.

Description

Technical Field

[0001] The present invention relates to the technical field of financial technology (Fintech), and in particular to a face attribute recognition method, client, device and medium based on federated learning. Background Art

[0002] With the development of computer technology, more and more technologies are being applied in the financial field. The traditional financial industry is gradually transforming into financial technology (Fintech). However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology.

[0003] The recognition of facial attributes includes the recognition of multiple attributes of the face, such as gender, age, appearance, whether wearing glasses, whether wearing a mask, etc. Existing facial attribute recognition solutions all use local facial image data to train models and then recognize facial attributes. However, in real-world scenarios, few companies or organizations have a large amount of facial attribute data, especially some sensitive attributes such as age. Therefore, the recognition accuracy of facial attributes by models trained with facial image data owned by each company is low, that is, the existing recognition accuracy of facial attributes is low. Summary of the invention

[0004] The main purpose of the present invention is to provide a method, client, device and medium for face attribute recognition based on federated learning, aiming to solve the problem of low recognition accuracy of face attributes.

[0005] To achieve the above object, the present invention provides a method for recognizing facial attributes based on federated learning, which is applied to a client. The method for recognizing facial attributes based on federated learning includes:

[0006] The client obtains a face image;

[0007] The facial attributes are obtained by identifying the facial image through a facial attribute recognition model in the client, wherein the facial attribute recognition model is obtained by longitudinal federated learning based on facial features provided by different clients;

[0008] Output the face attributes.

[0009] Optionally, before the step of obtaining the facial image by the client, the method further includes:

[0010] Obtaining facial features corresponding to each training sample, and sending each facial feature to the server;

[0011] Receiving the initial model and the accumulated gradient fed back by the server, wherein the server performs longitudinal federated learning on the facial features sent by each client to obtain alignment data, and obtains the initial model and the accumulated gradient according to the alignment data;

[0012] Setting the training parameters of the initial model according to the accumulated gradient to obtain a training model, and training the training model according to each of the training samples;

[0013] When the convergence value of the training model is less than a preset threshold, the training of the training model is stopped, and the training model whose training is stopped is saved as a face attribute recognition model.

[0014] Optionally, the step of receiving the initial model and accumulated gradient fed back by the server includes:

[0015] Receiving the initial model fed back by the server, and identifying each of the training samples using the initial model to obtain each attribute recognition result;

[0016] Sending each of the attribute recognition results to a server, wherein the server determines a cumulative gradient according to each of the attribute recognition results and an attribute data set corresponding to each of the attribute recognition results, and the alignment data includes the attribute data set;

[0017] Receive the accumulated gradient sent by the server.

[0018] Optionally, after the step of sending each of the attribute recognition results to the server, the method further includes:

[0019] Receiving the loss function fed back by the server, wherein the server determines each corresponding loss function according to the attribute recognition result corresponding to each client and the attribute data set corresponding to the attribute recognition result;

[0020] Determine a gradient according to the loss function and each of the training samples, and send the gradient to the server, wherein the server calculates a cumulative gradient according to the gradient sent by each of the clients;

[0021] Execute the step of receiving the accumulated gradient sent by the server.

[0022] Optionally, the step of receiving the loss function fed back by the server includes:

[0023] determining the type of the initial model;

[0024] Sending the type of the initial model to the server, wherein the server determines the loss function corresponding to the client according to the type of the initial model, the attribute recognition result corresponding to the client, and the attribute value range corresponding to the attribute recognition result;

[0025] The loss function fed back by the server is received.

[0026] Optionally, the step of training the training model according to each of the training samples includes:

[0027] Receiving each label fed back by the server, wherein the server combines each facial feature belonging to the same user to obtain a label of a training sample corresponding to each facial feature;

[0028] Updating the label of the training sample according to the label;

[0029] The training model is trained according to each of the training samples and the updated label of each of the training samples.

[0030] Optionally, the step of obtaining facial features corresponding to each training sample includes:

[0031] Receiving the facial feature extraction model fed back by the server;

[0032] The facial features of each training sample are extracted by the facial feature extraction model to obtain the facial features corresponding to each training sample.

[0033] To achieve the above object, the present invention further provides a client, wherein the client is provided with a face attribute recognition model, and the client comprises:

[0034] Acquisition module, acquiring face image;

[0035] A recognition module, used to identify the face image by using the face attribute recognition model to obtain face attributes, wherein the face attribute recognition model is obtained by longitudinal federated learning based on face features provided by different clients;

[0036] An output module is used to output the facial attributes.

[0037] To achieve the above-mentioned purpose, the present invention also provides a device, which is provided with a facial attribute recognition model, and the device also includes a memory, a processor and a recognition program stored in the memory and executable on the processor, the facial attribute recognition model is connected to the processor, and when the recognition program is executed by the processor, the steps of the facial attribute recognition method as described above are implemented.

[0038] To achieve the above object, the present invention further provides a medium, on which a recognition program is stored, and when the recognition program is executed by a processor, the steps of the method for recognizing facial attributes as described above are implemented.

[0039] The present invention provides a method, client, device and medium for face attribute recognition based on federated learning. After acquiring a face image, the client recognizes the face image through a face attribute recognition model in the client to obtain face attributes, thereby outputting the recognized face attributes. Compared with the prior art that uses its own face image to train a face attribute recognition model, the present invention obtains a face attribute recognition model with high face attribute recognition accuracy by performing longitudinal federated learning on face features provided by different clients, thereby overcoming the defect in the prior art that the client uses its own face image to train a face attribute recognition model, resulting in a low recognition accuracy rate of the client's face attributes, and improving the recognition accuracy rate of the client's face attributes. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of the client / device structure of the hardware operating environment involved in the embodiment of the present invention;

[0041] Figure 2 This is a flow chart of a first embodiment of a method for face attribute recognition based on federated learning according to the present invention;

[0042] Figure 3 This is a flow chart of a second embodiment of the method for face attribute recognition based on federated learning of the present invention;

[0043] Figure 4 It is a flowchart of a third embodiment of the method for face attribute recognition based on federated learning of the present invention;

[0044] Figure 5 This is a flow chart of a fourth embodiment of a method for face attribute recognition based on federated learning according to the present invention;

[0045] Figure 6 Schematic diagram of the functional modules of the client of the present invention.

[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0048] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention.

[0049] The terminal in the embodiment of the present invention may be a client or device with a data processing function, and the client or device may be a backend server belonging to an enterprise, unit or organization.

[0050] like Figure 1 As shown, the client or device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005 and a face attribute recognition model 1006. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0051] Those skilled in the art will understand that Figure 1 The structure of the terminal shown in the figure does not constitute a limitation on the client or device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0052] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a recognition program.

[0053] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client and communicate data with the client; and the processor 1001 can be used to call the recognition program stored in the memory 1005 and perform the following operations:

[0054] The client obtains a face image;

[0055] The facial attributes are obtained by identifying the facial image through a facial attribute recognition model in the client, wherein the facial attribute recognition model is obtained by longitudinal federated learning based on facial features provided by different clients;

[0056] Output the face attributes.

[0057] In one embodiment, the processor 1001 may call the recognition program stored in the memory 1005, and further perform the following operations:

[0058] Obtaining facial features corresponding to each training sample, and sending each facial feature to the server;

[0059] Receiving the initial model and the accumulated gradient fed back by the server, wherein the server performs longitudinal federated learning on the facial features sent by each client to obtain alignment data, and obtains the initial model and the accumulated gradient according to the alignment data;

[0060] Setting the training parameters of the initial model according to the accumulated gradient to obtain a training model, and training the training model according to each of the training samples;

[0061] When the convergence value of the training model is less than a preset threshold, the training of the training model is stopped, and the training model whose training is stopped is saved as a face attribute recognition model.

[0062] In one embodiment, the processor 1001 may call the recognition program stored in the memory 1005, and further perform the following operations:

[0063] Receiving the initial model fed back by the server, and identifying each of the training samples using the initial model to obtain each attribute recognition result;

[0064] Sending each of the attribute recognition results to a server, wherein the server determines a cumulative gradient according to each of the attribute recognition results and an attribute data set corresponding to each of the attribute recognition results, and the alignment data includes the attribute data set;

[0065] Receive the accumulated gradient sent by the server.

[0066] In one embodiment, the processor 1001 may call the recognition program stored in the memory 1005, and further perform the following operations:

[0067] Receiving the loss function fed back by the server, wherein the server determines each corresponding loss function according to the attribute recognition result corresponding to each client and the attribute data set corresponding to the attribute recognition result;

[0068] Determine a gradient according to the loss function and each of the training samples, and send the gradient to the server, wherein the server calculates a cumulative gradient according to the gradient sent by each of the clients;

[0069] Execute the step of receiving the accumulated gradient sent by the server.

[0070] In one embodiment, the processor 1001 may call the recognition program stored in the memory 1005, and further perform the following operations:

[0071] determining the type of the initial model;

[0072] Sending the type of the initial model to the server, wherein the server determines the loss function corresponding to the client according to the type of the initial model, the attribute recognition result corresponding to the client, and the attribute value range corresponding to the attribute recognition result;

[0073] Receive the loss function fed back by the server.

[0074] In one embodiment, the processor 1001 may call the recognition program stored in the memory 1005, and further perform the following operations:

[0075] Receiving each label fed back by the server, wherein the server combines each facial feature belonging to the same user to obtain a label of a training sample corresponding to each facial feature;

[0076] Updating the label of the training sample according to the label;

[0077] The training model is trained according to each of the training samples and the updated label of each of the training samples.

[0078] In one embodiment, the processor 1001 may call the recognition program stored in the memory 1005, and further perform the following operations:

[0079] Receiving the facial feature extraction model fed back by the server;

[0080] The facial features of each training sample are extracted by the facial feature extraction model to obtain the facial features corresponding to each training sample.

[0081] Based on the hardware structure of the above-mentioned client / device, various embodiments of the face attribute recognition method based on federated learning of the present invention are proposed.

[0082] The present invention provides a method for face attribute recognition based on federated learning.

[0083] Reference Figure 2 , Figure 2 This is the first embodiment of the face attribute recognition method based on federated learning of the present invention. The face attribute recognition method includes:

[0084] Step S10, the client obtains a face image;

[0085] In this embodiment, the execution subject is a client or a device, which can be a client or a device with data processing functions, and can be a backend server of an enterprise, unit or organization. For ease of description, the following describes this embodiment in detail using the client as the execution subject.

[0086] The client can be connected to the image acquisition module of the front end. The image acquisition module can be a camera or a still camera. After acquiring the image, the image acquisition module sends the image to the client. The client analyzes the image to determine whether the image includes a face. If the image includes a face, the image is defined as a face image, and the face image needs to be recognized.

[0087] Step S20, identifying the face image by a face attribute recognition model in the client to obtain face attributes, wherein the face attribute recognition model is obtained by longitudinal federated learning based on face features provided by different clients;

[0088] The client is equipped with a face attribute recognition model, which is used to recognize the face image, thereby identifying the face attributes of the user to whom the face image belongs. Face attributes may include age, gender, appearance, headwear, and accessories.

[0089] The facial attribute recognition model is obtained by longitudinal federated learning of facial features provided by different clients. Longitudinal federated learning refers to the data alignment of facial features provided by different clients, so that the facial attributes of the facial features of the same user are complete, and the facial attribute values ​​are expanded according to the facial attribute values ​​of the facial features of different users. For example, the facial attributes of the facial features of user A provided by client A are age and gender, and the facial attributes of the facial features of user A provided by client B are age and appearance. After user A performs data alignment, the facial attributes of the facial features become age, gender and appearance. For another example, the minimum age of the facial attributes of all facial features is 25 years old, and the maximum is 80 years old. The aligned age is 25 to 80 years old. The client can obtain the facial attribute recognition model by training the preset model based on the aligned data and training samples.

[0090] The client inputs the face image into the face attribute recognition model. The face attribute recognition model extracts the features of the face image, identifies the attribute values ​​corresponding to the features, and then outputs the attribute values. The attribute values ​​and attributes constitute the face attributes. Face attributes include multiple attributes, such as age, gender, and appearance. In addition, the face attribute recognition model can directly identify specific areas of the face image, and thus obtain face attributes based on the recognition results of the specific areas. For example, the face attribute recognition model can identify the corners of the eyes and cheeks in the face image, and estimate the age of the user to whom the face image belongs through the wrinkles at the corners of the eyes and the fineness of the cheek skin; for another example, the face attribute recognition model can identify the upper lip and chin of the face image, and determine the gender of the user to whom the face image belongs based on the presence or absence of a beard or stubble.

[0091] Step S30, outputting the face attributes.

[0092] After recognizing the face image, the face attribute recognition model can output the face attributes, that is, the client obtains the face attributes output by the face attribute recognition model, and then outputs the face attributes. Face attributes include attributes and attribute values ​​corresponding to the attributes. Attributes are age, gender, appearance, etc., and attribute values ​​are numbers, number ranges, specific words, etc. For example, face attributes are: age 20 to 25 years old (number range), gender female (specific words), appearance 85 (number).

[0093] In the technical solution provided in this embodiment, after obtaining a face image, the client uses the face attribute recognition model in the client to identify the face image to obtain the face attributes, thereby outputting the identified face attributes. Compared with the prior art that uses its own face image to train the face attribute recognition model, the present invention obtains a face attribute recognition model with high face attribute recognition accuracy by performing longitudinal federated learning on the face features provided by different clients, thereby overcoming the defect in the prior art that the client uses its own face image to train the face attribute recognition model, resulting in a low recognition accuracy rate of the client's face attributes, and improving the recognition accuracy rate of the client's face attributes.

[0094] Reference Figure 3 , Figure 3 This is a second embodiment of the face attribute recognition method based on federated learning of the present invention. Based on the first embodiment, before step S10, it also includes:

[0095] Step S40, obtaining facial features corresponding to each training sample, and sending each facial feature to the server;

[0096] In this embodiment, each enterprise will send its own facial data to the server through the client. In order to ensure the confidentiality of the data, the client will not upload the facial image directly to the server. The server will send the facial feature extraction model to the client corresponding to each enterprise. After receiving the facial feature extraction model, the client will extract the facial features of the training samples to obtain the facial features corresponding to each training sample.

[0097] The client of company A extracts the facial features of the facial image using the facial feature extraction model. The client of company B extracts the facial features of the facial image using the facial feature extraction model. That is, the format of the facial features uploaded by each client to the server is the same. Among them, emb is feature embedding, y is a vector, and each dimension of the vector represents a facial attribute. If the facial data does not have a certain facial attribute (for example, company A does not have the age of the user), the dimension is represented by -1. By extracting facial features in this way, the format of the facial features provided by each client is the same, so that the server does not need to process the facial features to unify the format of each facial feature, reducing the workload of the server. At the same time, the client only uploads facial features to avoid the problem of privacy leakage caused by uploading facial images.

[0098] Step S50, receiving the initial model and accumulated gradient fed back by the server, wherein the server performs longitudinal federated learning on the facial features sent by each client to obtain aligned data, and obtains the initial model and accumulated gradient according to the aligned data;

[0099] After receiving each facial feature, the service segment will perform vertical federated learning on each facial feature. Vertical federated learning can be regarded as type alignment and value alignment of each facial feature to obtain aligned data. Type alignment refers to the union of all facial attributes. For example, the facial attribute type owned by company A is (gender, age), and the facial attribute type owned by company B is (age, expression). The union of (gender, age) and (age, expression) is used to obtain various facial attribute types (gender, age, expression). After completing type alignment to obtain various facial attribute types, it is necessary to align the attribute values ​​of a single facial attribute type. For example, if the facial attribute type is age, the age corresponding to the facial features provided by company A is 15 to 50 years old, and the age corresponding to the facial features provided by company B is 25 to 80 years old, then the union of 15 to 50 years old and 25 to 80 years old is taken to obtain the attribute value range corresponding to the facial attribute type of age: 15 to 80 years old. By analogy, the attribute value range corresponding to each facial attribute type is obtained. That is, the alignment data includes multiple facial attribute types and the attribute value range corresponding to each facial attribute type.

[0100] Specifically, the y in the facial feature carries the attribute category. The server determines the various facial attribute types according to the attribute types corresponding to each facial feature, that is, aligns the categories of each facial feature. The server then determines the attribute value range corresponding to each facial attribute category based on the attribute values ​​corresponding to each attribute category of the same facial attribute type. The attribute value corresponding to the attribute category is the specific value on each dimension of the vector y in the facial feature.

[0101] After obtaining the alignment data, the server can determine the cumulative gradient and the initial model according to the alignment data. The initial model is actually generated by the server according to the alignment data. Specifically, the server is provided with a model, and the model structure and the output face attribute format of the model are defined through various face attribute types and each face attribute type, so as to obtain the initial model.

[0102] The server may store a default cumulative gradient in advance, and the server may determine a correction value according to the alignment data, thereby correcting the default cumulative gradient according to the correction value to obtain a corrected cumulative gradient. The corrected cumulative gradient is the cumulative gradient obtained according to the alignment data.

[0103] After obtaining the initial model and accumulated gradients, the server sends the initial model and accumulated gradients to each client.

[0104] Step S60, setting the training parameters of the initial model according to the accumulated gradient to obtain a training model, and training the training model according to each of the training samples;

[0105] Step S70, when the convergence value of the training model is less than a preset threshold, stop training the training model, and save the training model that has stopped training as a face attribute recognition model.

[0106] After receiving the initial model and the accumulated gradient, since the training parameters of the initial model have not been set, the client can set the training parameters of the initial model according to the accumulated gradient to obtain the training model, thereby training the training model through the training samples in the training set, and then training to obtain the face attribute recognition model. After completing the training of the training model with all training samples, if the convergence value of the network of the training model is greater than or equal to the preset threshold, it can be determined that the training model has not converged, and the client continues to send facial features to the server, so that the server repeats steps S40-S60 to perform the next iteration of the model. If the convergence value is less than the preset threshold, the training model is saved as a face attribute recognition model.

[0107] In the technical solution provided in this embodiment, the client obtains the facial features corresponding to each training sample, and sends each facial feature to the server. The server performs longitudinal federated learning on the facial features sent by each client to obtain alignment data, thereby obtaining an initial model and cumulative gradients based on the alignment data, and sends the initial model and cumulative gradients to each client. The client then sets training parameters for the initial model based on the cumulative gradient to obtain a training model, thereby training the training model based on each training sample to obtain a facial attribute recognition model. Since the initial model and the training parameters of the initial model are determined by longitudinal federated learning of facial features provided by different clients, the trained facial attribute recognition model can recognize more facial attribute types and can recognize more accurate facial attribute values, thereby improving the recognition accuracy of the facial attribute recognition model for facial attributes.

[0108] Reference Figure 4 , Figure 4 This is the third embodiment of the face attribute recognition method based on federated learning of the present invention. Based on the second embodiment, step S50 includes:

[0109] Step S51, receiving the initial model fed back by the server, and identifying each of the training samples using the initial model to obtain each attribute recognition result;

[0110] Step S52, sending each of the attribute recognition results to a server, wherein the server determines a cumulative gradient according to each of the attribute recognition results and an attribute data set corresponding to each of the attribute recognition results, and the alignment data includes the attribute data set;

[0111] Step S53, receiving the accumulated gradient sent by the server.

[0112] In this embodiment, the cumulative gradient is generated by the server according to the characteristics of the client itself, and the characteristics of the client itself are represented by the training samples of the client.

[0113] Specifically, after receiving the initial model, the client needs to train the initial model with its own face image. Before training, the training parameters of the initial model have not been set. To this end, the client first determines each training sample in the training set, where the training sample is a face image, and performs attribute recognition on each training sample through the initial model to obtain the attribute recognition result corresponding to each training sample. The attribute recognition result is represented as in, represents a face image, Represents the attribute identification function. In order to ensure the confidentiality of the data, Delete it and then upload the attribute recognition results to the server.

[0114] After receiving the attribute recognition results corresponding to the facial features uploaded by each client, the server extracts the features from the attribute recognition results. After the server aligns the facial features, the resulting data is represented as follows: The server searches for the aligned data and attribute recognition results with the same features, and can determine the attribute data set corresponding to the attribute recognition result. For example, the attribute recognition result is The server finds the alignment data corresponding to the attribute recognition result Then the attribute data set corresponding to the attribute recognition result is The server can determine the cumulative gradient based on each attribute recognition result and the attribute data set corresponding to the attribute recognition result. The server can determine the training parameter setting direction of the initial model in the client based on each attribute recognition result of the client and the attribute data set corresponding to the attribute recognition result. The server can determine the cumulative gradient by combining the training parameter setting directions of all clients.

[0115] The server then sends the accumulated gradients to each client, so that each client sets the training parameters of the initial model based on the accumulated gradients.

[0116] In the technical solution provided in this embodiment, after receiving the initial model fed back by the server, the client uses the initial model to identify each training sample to obtain each attribute recognition result, and then sends the attribute recognition result to the server. The server determines the cumulative gradient based on each attribute recognition result and the attribute data set corresponding to each attribute recognition result, so that the client can accurately set the training parameters of the initial model based on the cumulative gradient sent by the server.

[0117] Reference Figure 5 , Figure 5 This is the fourth embodiment of the face attribute recognition method based on federated learning of the present invention. Based on the third embodiment, after step S52, it also includes:

[0118] Step S54, receiving the loss function fed back by the server, wherein the server determines each corresponding loss function according to the attribute recognition result corresponding to each client and the attribute data set corresponding to the attribute recognition result;

[0119] Step S55, determining a gradient according to the loss function and each of the training samples, and sending the gradient to the server, wherein the server calculates a cumulative gradient according to the gradient sent by each of the clients;

[0120] Step S56, executing the step of receiving the accumulated gradient sent by the server.

[0121] In this embodiment, the setting direction of the training parameters of the initial model in the client can be characterized by the loss function of the initial model in the client.

[0122] Specifically, the server calculates the loss function corresponding to each client according to the attribute recognition result corresponding to each client and the attribute data set corresponding to the attribute recognition result.

[0123] The server can obtain the loss function by calculating the cross entropy or the mean square error (mse). The loss function L calculated by cross entropy can be obtained by referring to the formula: The loss function L calculated by mse can refer to the formula The y in both formulas i Characterized as a collection of attribute data.

[0124] Of course, the initial model of each client can be a default type. In this case, the default loss function calculation formula of the server client determines the client's loss function.

[0125] The server sends the calculated loss function to the corresponding client, and the client calculates the gradient based on the loss function and each sample in the training set. The samples in the training set are the facial features corresponding to the face image. The client sends the calculated gradient to the server, and the server aggregates each gradient to obtain the cumulative gradient.

[0126] In the technical solution provided in this embodiment, the client receives the loss function fed back by the server, and determines the gradient based on the loss function and each training sample, and then sends the gradient to the server, so that the server combines the gradients sent by each client to obtain the cumulative gradient, and then sends the cumulative gradient to the client, so that the client sets the training parameters of the initial model based on the characteristics of other clients.

[0127] In one embodiment, the step of receiving the loss function fed back by the server includes:

[0128] Step S1, determining the type of the initial model;

[0129] Step S2, sending the type of the initial model to the server, wherein the server determines the loss function corresponding to the client according to the type of the initial model, the attribute recognition result corresponding to the client, and the attribute value range corresponding to the attribute recognition result;

[0130] Step S3, receiving the loss function fed back by the server.

[0131] The calculation of the loss function is related to the type of the initial model, which includes regression model and classification model. That is, the client first determines the type of the initial model, that is, whether to use the initial model for classification task or regression task. The client then sends the type of the initial model to the server. The server determines the loss function corresponding to the client based on the type of the initial model, the attribute recognition result corresponding to the client, and the attribute data set corresponding to the attribute recognition result. Specifically, if the initial model is a classification model, it is necessary to calculate the cross entropy to obtain the loss function. If the initial model is a regression model, it is necessary to calculate the mean square error mse to obtain the loss function. The loss function L calculated by cross entropy can be referred to the formula: The loss function L obtained by calculating mse can refer to the formula The y in both formulas i Characterized as a collection of attribute data.

[0132] In this embodiment, the server can calculate the loss function for the client through the type of the initial model, thereby setting matching training parameters for the initial model.

[0133] In one embodiment, the step of training the training model according to each of the training samples includes:

[0134] Step C1, receiving each label fed back by the server, wherein the server combines each facial feature belonging to the same user to obtain a label of a training sample corresponding to each facial feature;

[0135] Step C2, updating the label of the training sample according to the label;

[0136] Step C3: training the training model according to each of the training samples and the updated label of each of the training samples.

[0137] Different clients represent different companies, and different companies may have facial images of the same user. For example, both companies A and B have facial images of Zhang San, but the facial attributes of Zhang San owned by companies A and B may be different. For example, company A has Zhang San’s age, while company B has Zhang San’s expression. Therefore, the server needs to align the facial features belonging to the same user. If two facial images belong to the same user, the distance between the facial features extracted from the two facial images will be small. In this regard, the server calculates the distance between each facial feature, and the distance can be calculated using the following formula: Among them, D represents the dimension of feature emb, dist(emb A ,emb B) is the distance between the facial features provided by enterprise A and the facial features provided by enterprise B. The server determines the facial features whose distance is less than the preset distance as facial features belonging to the same user.

[0138] The server merges the facial features of the same user to obtain the labels of the training samples corresponding to each facial feature. For example, the training samples in Company A originally had only one label, 23 years old. After merging the facial features of the users corresponding to the training samples, the labels corresponding to the training samples are 23 years old and 85 in appearance.

[0139] After obtaining the label corresponding to each facial feature, the server sends the label to the client corresponding to the facial feature, so that the client updates the label of the training sample corresponding to the facial feature. For example, if the label corresponds to the facial features provided by company A, company B, and company D, the server will send the label to the clients corresponding to the three companies A, B, and D. It can be understood that the label is actually the union of each facial feature. Assuming that the label is the combination of the first facial feature of company A and the second facial feature of company B, the label can be expressed as When the server sends a tag to the client of company A, it will change the tag to This hides from company A the fact that company B has the same user’s facial data, and company A uses the features in the tag to identify the user. You can determine y M Belongs to the first training sample, thus updating the label of the first training sample.

[0140] After obtaining each label, the server sends each label to the corresponding client. After receiving each label fed back by the server, the client updates the label of the training sample according to the label, and then trains the training model according to the training sample and the updated label of the training sample to train the obtained face attribute recognition model.

[0141] Since the updated labels correspond to more complete facial attributes, the client can obtain a facial attribute recognition model with higher recognition accuracy based on training samples, labels corresponding to the training samples, the initial model and cumulative gradient training.

[0142] The invention also provides a client.

[0143] Reference Figure 6 , Figure 6 Schematic diagram of the functional modules of the client of the present invention.

[0144] like Figure 6 As shown, the client includes:

[0145] An acquisition module 10 is used to acquire a face image;

[0146] A recognition module 20, configured to recognize the face image and obtain face attributes by using the face attribute recognition model, wherein the face attribute recognition model is obtained by longitudinal federated learning based on face features provided by different clients;

[0147] The output module 30 is used to output the facial attributes.

[0148] In one embodiment, the client further includes a receiving module, a setting module and a stopping module:

[0149] An acquisition module 10 is used to acquire the facial features corresponding to each training sample and send each facial feature to the server;

[0150] A receiving module, used to receive the initial model and accumulated gradient fed back by the server, wherein the server performs longitudinal federated learning on the facial features sent by each client to obtain alignment data, and obtains the initial model and accumulated gradient according to the alignment data;

[0151] A setting module, used for setting the training parameters of the initial model according to the accumulated gradient to obtain a training model, and training the training model according to each of the training samples;

[0152] The stopping module is used to stop training the training model when the convergence value of the training model is less than a preset threshold, and save the training model that has stopped training as a face attribute recognition model.

[0153] In one embodiment, the client further includes:

[0154] A receiving module, used to receive the initial model fed back by the server, and identify each of the training samples through the initial model to obtain each attribute recognition result;

[0155] A sending module, used to send each of the attribute recognition results to a server, wherein the server determines a cumulative gradient according to each of the attribute recognition results and an attribute data set corresponding to each of the attribute recognition results, and the alignment data includes the attribute data set;

[0156] The receiving module is used to receive the accumulated gradient sent by the server.

[0157] In one embodiment, the client further includes:

[0158] A receiving module, configured to receive the loss function fed back by the server, wherein the server determines each corresponding loss function according to the attribute recognition result corresponding to each client and the attribute data set corresponding to the attribute recognition result;

[0159] A determination module, configured to determine a gradient according to the loss function and each of the training samples, and send the gradient to the server, wherein the server calculates a cumulative gradient according to the gradient sent by each of the clients;

[0160] An execution module is used to execute the step of receiving the accumulated gradient sent by the server.

[0161] In one embodiment, the client further includes:

[0162] A determination module, used to determine the type of the initial model;

[0163] A sending module, used for sending the type of the initial model to the server, wherein the server determines the loss function corresponding to the client according to the type of the initial model, the attribute recognition result corresponding to the client, and the attribute value range corresponding to the attribute recognition result;

[0164] The receiving module is used to receive the loss function fed back by the server.

[0165] In one embodiment, the client further includes:

[0166] A receiving module, configured to receive the labels fed back by the server, wherein the server combines the facial features of the same user to obtain a label of a training sample corresponding to each facial feature;

[0167] An updating module, used for updating the label of the training sample according to the label;

[0168] The training module is used to train the training model according to each of the training samples and the updated label of each of the training samples.

[0169] In one embodiment, the client further includes:

[0170] A receiving module, used for receiving the facial feature extraction model fed back by the server;

[0171] The extraction module is used to extract the facial features of each training sample through the facial feature extraction model to obtain the facial features corresponding to each training sample.

[0172] Among them, the functional implementation of each module in the above-mentioned client corresponds to the various steps in the above-mentioned face attribute recognition method based on federated learning, and its functions and implementation processes will not be repeated here one by one.

[0173] The present invention also provides a medium on which a recognition program is stored. When the recognition program is executed by a processor, the steps of the face attribute recognition method based on federated learning as described in any of the above embodiments are implemented.

[0174] The specific embodiments of the medium of the present invention are basically the same as the embodiments of the above-mentioned face attribute recognition method based on federated learning, and will not be described in detail here.

[0175] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0176] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0177] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0178] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A face attribute recognition method based on federated learning, characterized in that: The face attribute recognition method based on federated learning is applied to a client, and the face attribute recognition method based on federated learning includes: The client obtains a face image; The facial attributes are obtained by identifying the facial image through the facial attribute recognition model in the client, and the facial attribute recognition model is obtained by longitudinal federated learning according to the facial features provided by different clients; wherein the facial attributes include attributes and attribute values ​​corresponding to the attributes, and the attributes include age, gender, appearance, headwear, and accessories; longitudinal federated learning refers to aligning the facial features provided by different clients, so that the facial attributes of the facial features of the same user are completed, and the facial attribute values ​​are expanded according to the facial attribute values ​​of the facial features of different users; Outputting the facial attributes; Before the step of obtaining the face image by the client, the method further includes: Obtaining facial features corresponding to each training sample, and sending each facial feature to the server; Receiving the initial model and the accumulated gradient fed back by the server, wherein the server performs longitudinal federated learning on the facial features sent by each client to obtain alignment data, and obtains the initial model and the accumulated gradient according to the alignment data; Setting the training parameters of the initial model according to the accumulated gradient to obtain a training model, and training the training model according to each of the training samples; When the convergence value of the training model is less than a preset threshold, the training of the training model is stopped, and the training model whose training is stopped is saved as a face attribute recognition model.

2. The method for face attribute recognition based on federated learning according to claim 1, characterized in that: The step of receiving the initial model and accumulating gradients fed back by the server comprises: Receiving the initial model fed back by the server, and identifying each of the training samples using the initial model to obtain each attribute recognition result; Sending each of the attribute recognition results to a server, wherein the server determines a cumulative gradient according to each of the attribute recognition results and an attribute data set corresponding to each of the attribute recognition results, and the alignment data includes the attribute data set; Receive the accumulated gradient sent by the server.

3. The method for face attribute recognition based on federated learning as claimed in claim 2, characterized in that: After the step of sending each of the attribute recognition results to the server, the method further includes: Receiving the loss function fed back by the server, wherein the server determines each corresponding loss function according to the attribute recognition result corresponding to each client and the attribute data set corresponding to the attribute recognition result; Determine a gradient according to the loss function and each of the training samples, and send the gradient to the server, wherein the server calculates a cumulative gradient according to the gradient sent by each of the clients; Execute the step of receiving the accumulated gradient sent by the server.

4. The method for face attribute recognition based on federated learning as claimed in claim 3, characterized in that: The step of receiving the loss function fed back by the server comprises: determining the type of the initial model; Sending the type of the initial model to the server, wherein the server determines the loss function corresponding to the client according to the type of the initial model, the attribute recognition result corresponding to the client, and the attribute value range corresponding to the attribute recognition result; Receive the loss function fed back by the server.

5. The method for face attribute recognition based on federated learning according to any one of claims 1 to 4, characterized in that: The step of training the training model according to each of the training samples comprises: Receiving each label fed back by the server, wherein the server combines each facial feature belonging to the same user to obtain a label of a training sample corresponding to each facial feature; Updating the label of the training sample according to the label; The training model is trained according to each of the training samples and the updated label of each of the training samples.

6. The method for face attribute recognition based on federated learning according to any one of claims 1 to 4, characterized in that: The step of obtaining the facial features corresponding to each training sample comprises: Receiving the facial feature extraction model fed back by the server; The facial features of each training sample are extracted by the facial feature extraction model to obtain the facial features corresponding to each training sample.

7. A client, characterized in that: The client is provided with a face attribute recognition model, and the client comprises: An acquisition module is used to acquire a face image. The acquisition module is also used to acquire the face features corresponding to each training sample and send each face feature to the server. A recognition module, used for recognizing the face image through the face attribute recognition model to obtain face attributes, wherein the face attribute recognition model is obtained by longitudinal federated learning based on face features provided by different clients; wherein the face attributes include attributes and attribute values ​​corresponding to the attributes, and the attributes include age, gender, appearance, headwear, and accessories; longitudinal federated learning refers to aligning the face features provided by different clients, so that the face attributes of the face features of the same user are completed, and the face attribute values ​​are expanded according to the face attribute values ​​of the face features of different users; An output module, used for outputting the facial attributes; A receiving module, used to receive the initial model and accumulated gradient fed back by the server, wherein the server performs longitudinal federated learning on the facial features sent by each client to obtain alignment data, and obtains the initial model and accumulated gradient according to the alignment data; A setting module, used for setting the training parameters of the initial model according to the accumulated gradient to obtain a training model, and training the training model according to each of the training samples; The stopping module is used to stop training the training model when the convergence value of the training model is less than a preset threshold, and save the training model that has stopped training as a face attribute recognition model.

8. A device, characterized in that: The device is provided with a facial attribute recognition model, and the device also includes a memory, a processor, and a recognition program stored in the memory and executable on the processor. The facial attribute recognition model is connected to the processor, and when the recognition program is executed by the processor, the steps of the facial attribute recognition method as described in any one of claims 1 to 6 are implemented.

9. A medium, characterized in that The medium stores a recognition program, and when the recognition program is executed by the processor, the steps of the method for recognizing facial attributes according to any one of claims 1 to 6 are implemented.

Images