Facial attribute detection optimization method, device, equipment, storage medium and product

Abstract

The present application discloses a facial attribute detection optimization method, apparatus, device, storage medium, and program product. The facial attribute detection optimization method includes the following steps: obtaining training sample facial images and attribute labels; performing local model training based on the training sample facial images and attribute labels to determine local model update data corresponding to a common feature extraction model; sending the local model update data to a second device so that the second device can aggregate the local model update data sent by each first device to obtain global model update data; receiving the global model update data sent by the second device, updating the common feature extraction model based on the global model update data, and returning to execute the step of obtaining training sample facial images and attribute labels until a preset federated training end condition is met. The present application solves the technical problem of low accuracy in attribute detection of facial images by federated modeling in the prior art.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology in financial technology (Fintech), and in particular to a facial attribute detection optimization method, device, equipment, storage medium and program product. Background Art

[0002] With the continuous development of financial technology, especially Internet technology finance, more and more technologies (such as distributed, artificial intelligence, etc.) are being applied in the financial field, but the financial industry also puts forward higher requirements for technology.

[0003] Attribute detection on facial images is a widely used function. It involves determining certain attributes of a person based on facial images. Common attribute detection methods include age, gender, skin, and mask detection. Currently, federated learning technology is used for federated modeling, effectively leveraging data from various parties for model training and improving attribute detection on facial images. However, different institutions have different requirements for attribute detection on facial images and differ in their equipment conditions. In this case, federated learning requires a unified model structure for all parties, which may result in poor model performance for some participants, leading to reduced accuracy in attribute detection on facial images. Summary of the Invention

[0004] The main purpose of this application is to provide a facial attribute detection optimization method, device, equipment, storage medium and program product, aiming to solve the technical problem of low accuracy of attribute detection of facial images through federated modeling in the existing technology.

[0005] To achieve the above objectives, the present application provides a facial attribute detection optimization method, which is applied to any one of multiple first devices in a federated learning system, where an individual feature extraction model, a common feature extraction model, and an attribute detection model are deployed on the first device; the facial attribute detection optimization method comprises the following steps:

[0006] Obtain training sample face images and attribute labels;

[0007] Performing local model training on the individual feature extraction model, the common feature extraction model, and the attribute detection model according to the training sample face image and the attribute label, and determining local model update data corresponding to the common feature extraction model;

[0008] Sending the local model update data to a second device, so that the second device aggregates the local model update data sent by each of the first devices to obtain global model update data, and sending the global model update data to each of the first devices;

[0009] Receive the global model update data sent by the second device, update the common feature extraction model based on the global model update data, and return to execute the step of obtaining the training sample face image and attribute label until the preset federated training end condition is met.

[0010] The present application also provides a facial attribute detection optimization method, which is applied to a third device, wherein a personality feature extraction model, a common feature extraction model, and an attribute detection model are deployed on the third device, wherein the personality feature extraction model, the common feature extraction model, and the attribute detection model are trained using the facial attribute detection optimization method described above; the facial attribute detection optimization method comprises the following steps:

[0011] Get the face image to be detected;

[0012] Extracting features from the face image to be detected using a personality feature extraction model to obtain personality detection features, and extracting features from the face image to be detected using a common feature extraction model to obtain common detection features;

[0013] The individuality detection feature and the commonality detection feature are input into an attribute detection model to obtain an attribute detection result.

[0014] The present application further provides a facial attribute detection and optimization device, which is applied to any one of multiple first devices in a federated learning system, wherein an individual feature extraction model, a common feature extraction model, and an attribute detection model are deployed on the first device, and the facial attribute detection and optimization device includes:

[0015] Acquisition module, used to obtain training sample face images and attribute labels;

[0016] A local training module is used to perform local model training on the individual feature extraction model, the common feature extraction model, and the attribute detection model according to the training sample face image and the attribute label, and determine the local model update data corresponding to the common feature extraction model;

[0017] a sending module, configured to send the local model update data to a second device, so that the second device aggregates the local model update data sent by each of the first devices to obtain global model update data, and sends the global model update data to each of the first devices;

[0018] An update module is used to receive the global model update data sent by the second device, update the common feature extraction model based on the global model update data, and return to execute the step of obtaining the training sample face image and attribute label until the preset federated training end condition is met.

[0019] The present application further provides a facial attribute detection optimization device, which is applied to a third device. An individual feature extraction model, a common feature extraction model, and an attribute detection model are deployed on the third device. The individual feature extraction model, the common feature extraction model, and the attribute detection model are trained using the facial attribute detection optimization method described above. The facial attribute detection optimization device includes:

[0020] An acquisition module is used to acquire the face image to be detected;

[0021] A feature extraction module is used to extract features from the face image to be detected using an individual feature extraction model to obtain individual detection features, and to extract features from the face image to be detected using a common feature extraction model to obtain common detection features;

[0022] The detection module is used to input the individual detection feature and the common detection feature into an attribute detection model to obtain an attribute detection result.

[0023] The present application also provides an electronic device, which is a physical device, and includes: a memory, a processor, and a program of the facial attribute detection optimization method stored in the memory and runnable on the processor. When the program of the facial attribute detection optimization method is executed by the processor, the steps of the facial attribute detection optimization method as described above can be implemented.

[0024] The present application also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores a program for implementing a facial attribute detection optimization method. When the program of the facial attribute detection optimization method is executed by a processor, the steps of the facial attribute detection optimization method as described above are implemented.

[0025] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned facial attribute detection optimization method when executed by a processor.

[0026] The present application provides a method, apparatus, device, storage medium and program product for optimizing face attribute detection, wherein the method for optimizing face attribute detection is applied to any one of a plurality of first devices in a federated learning system, wherein a personality feature extraction model, a common feature extraction model and an attribute detection model are deployed on the first device, and by obtaining training sample face images and attribute labels, local model training is performed on the personality feature extraction model, the common feature extraction model and the attribute detection model according to the training sample face images and the attribute labels, and local model update data corresponding to the common feature extraction model is determined, thereby achieving the purpose of local model training on the personality feature extraction model, the common feature extraction model and the attribute detection model based on the private training sample data of the first device, and then the local model update data is sent to the second device for the second device to send to each of the first devices. The local model update data of each of the above-mentioned local models are aggregated to obtain the global model update data, and the global model update data is sent to each of the above-mentioned first devices, thereby achieving the purpose of jointly training the common feature extraction model by each of the above-mentioned first devices participating in the federated learning. More training sample data can be used for training to improve the accuracy of the common feature extraction model, thereby improving the accuracy of the common feature extraction in the face image, thereby improving the accuracy of the attribute detection of the face image, and then by receiving the global model parameters sent by the second device, the common feature extraction model is updated based on the global model parameters, and the step of obtaining the training sample face image and attribute label is returned to execute until the preset federated training end condition is met, thereby achieving the training of the individual feature extraction model using the private training sample data of the first device, and the joint training of the common feature extraction model by multiple first devices. When the attributes to be detected by each first device are different, because the sample data used for detection are all face images, even if the attributes to be detected are different or the requirements for model performance are different, the sample features required for detection are often partially the same. Therefore, the common feature extraction model can be used to extract the sample features of the intersection part, and the individual feature extraction model can be used to extract the sample features of the difference part. In this way, compared with the method of training the same feature extraction model for detecting different attributes through federated learning for feature extraction, the present application can realize layered feature extraction and solve the aggregation problem caused by model inconsistency. The common feature extraction model can be trained through federated learning to mine the data commonalities between the first devices and maintain consistency for aggregation, effectively improving the effect of the common feature extraction model. The model structure, training data, etc. of the individual feature extraction model can be flexibly determined according to the own data of each first device and the actual needs of detecting different attributes, focusing on mining the individual feature extraction of local data without being affected by federated learning. Therefore, the effect of the individual feature extraction model will also be better.This overcomes the technical drawback that federated learning requires a unified model structure for all parties, which can lead to poor model performance for some participants and reduced accuracy in facial attribute detection. This improves the accuracy of attribute detection for facial images through federated modeling. Furthermore, only a common feature extraction model needs to be trained on a federated basis, reducing data transmission and improving the training efficiency of federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0029] Figure 1 This is a flow chart of the first embodiment of the applicant's facial attribute detection optimization method;

[0030] Figure 2 This is a schematic diagram of a scenario in which a federated learning system can be implemented in an embodiment of the present application;

[0031] Figure 3 This is a schematic diagram of a possible implementation scenario of step S20 in the embodiment of the present application;

[0032] Figure 4 This is a schematic diagram of a possible implementation scenario of step S40 in the embodiment of the present application;

[0033] Figure 5 This is a flow chart of the second embodiment of the applicant's facial attribute detection optimization method;

[0034] Figure 6 Schematic diagram of the model structure of multi-attribute detection in the embodiment of the present application;

[0035] Figure 7 This is a flow chart of the third embodiment of the applicant's facial attribute detection optimization method;

[0036] Figure 8 This is a schematic diagram of the structure of the face attribute detection and optimization device in an embodiment of the present application;

[0037] Figure 9Schematic diagram of the device structure of the hardware operating environment involved in the face attribute detection optimization method in the embodiment of the present application.

[0038] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0039] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0040] Example 1

[0041] The present application provides a method for optimizing face attribute detection. In the first embodiment of the present application's method for optimizing face attribute detection, refer to Figure 1 The facial attribute detection optimization method is applied to any one of multiple first devices in a federated learning system, where an individual feature extraction model, a common feature extraction model, and an attribute detection model are deployed on the first device; the facial attribute detection optimization method includes the following steps:

[0042] Step S10, obtaining training sample face images and attribute labels;

[0043] The executor of the method of this embodiment can be a facial attribute detection optimization device, or a facial attribute detection optimization terminal device or server. This embodiment takes a facial attribute detection optimization device as an example, and the facial attribute detection optimization device can be integrated into a terminal device such as a smart phone, computer, etc. with data processing capabilities.

[0044] Attribute detection of facial images is a widely used function. It involves determining certain attributes of a person based on facial images. Common attribute detection methods include age, gender, skin, and mask detection. Different organizations have different requirements for attribute detection. For example, Company A may want to detect age and gender, Company B may want to detect whether a person is wearing a mask, and Company C may want to detect gender and mask detection. Furthermore, different organizations have different equipment conditions, resulting in different requirements for model performance. For example, some organizations may require faster response times, while others may prioritize higher accuracy.

[0045] The model for attribute detection of facial images can usually be abstracted into two parts, namely, the feature extraction layer and the task layer. The task layer can be determined based on the attributes that need to be detected. If different institutions need to detect different attributes, the task layer is often different. For example, if it is gender detection, the task layer is a binary classification network. If it is age detection, it can be constructed as a regression model, etc.

[0046] The sample data used for detection are all facial images. Even if the attributes to be detected are different or the requirements for model performance vary, the sample features required for detection are often partially the same. In this case, if each institution conducts training independently, it will be time-consuming and labor-intensive, requiring repeated execution of many of the same tasks. Moreover, the number and richness of training samples are not as good as those of federated learning. Therefore, multiple institutions can be combined to perform horizontal federated training on the feature extraction layer. Horizontal federated learning involves conducting joint machine learning on the data where there is significant overlap in the data features of each participant but less overlap in the users. This method involves extracting the data where the data features of the participants are the same but the users are not exactly the same. By expanding the number of samples, the accuracy of attribute detection based on facial images can be improved.

[0047] However, because different institutions need to detect different attributes, the sample features required to accurately detect different attributes are often not exactly the same. For example, to detect whether a mask is worn, the main features of the mouth, nose, and ear areas may be extracted, while to detect age, the features of the mouth, nose, eyes, forehead, and hair areas may need to be extracted. Federated learning requires that the model structure of all parties be unified; different institutions have different requirements for model performance, and the feature extraction models they choose may also be different. For example, some institutions have requirements for response speed, so the feature extraction model may use a lightweight model, such as Mobilenet; some institutions may have higher requirements for accuracy, so the feature extraction model may use a feature extraction model such as Resnet.

[0048] Therefore, using federated learning to train feature extraction models from different institutions will make it difficult to meet the actual needs of each participant. This may lead to reduced attribute detection accuracy for some participants, or the equipment conditions of some participants may not meet the requirements of model training and application. Furthermore, restrictions on participants may also result in a small number of participants being able to participate in federated learning. Therefore, because different participants require different detection attributes, forcing them to sample the same model structure may not be the optimal choice.

[0049] In this embodiment, it should be noted that the facial attribute detection optimization method is applied to any one of multiple first devices in a federated learning system, where the first device refers to a participant in federated learning. A horizontal federated learning system includes multiple participants, namely multiple first devices, and may also include a coordinator. The second device may be the coordinator or one of the participants, and the specific method may be determined based on actual circumstances, which is not limited in this embodiment. A feature extraction model and an attribute detection model are deployed on the first device. The feature extraction model is connected in series with the attribute detection model. The feature extraction model includes an individual feature extraction model and a common feature extraction model. The individual feature extraction model can be connected in series or in parallel with the common feature extraction model. In one practicable method, the common feature extraction model can first be used to extract features from the sample data to obtain common features. The individual feature extraction model can then be used to extract features from the common features to obtain individual features. The individual features can then be input into the attribute detection model for attribute detection.

[0050] The common feature extraction models deployed on each first device are the same. Before the federated training begins, the second device can send the same common feature extraction model to each first device, or each first device can establish an initial common feature extraction model on its own after negotiation. The training of the common feature extraction model will be carried out using the training sample face images of each participant including itself, so it is suitable for mining the common sample features required for different attribute detections; the individual feature extraction model and the attribute detection model are only trained based on the private local data of the first device, so the individual feature extraction model is suitable for extracting common features. In addition to the characteristics of the sexual sample, there are specific characteristics that have little correlation with the attributes that other participants need to detect, but are closely related to the attributes that one party needs to detect. For example, the first device E1 needs to detect whether a mask is worn, so it needs to extract the features of the mouth, nose, and ear areas. The second device E2 needs to detect age, so it needs to extract the features of the mouth, nose, eyes, forehead, and hair areas. Therefore, a common feature extraction model that extracts features of the mouth and nose areas can be trained through federated learning, while the individual feature extraction model deployed on the first device E1 only needs to extract features of the ear area, and the individual feature extraction model deployed on the second device E2 only needs to extract features of the eyes, forehead, and hair areas.

[0051] In one practicable manner, referring to Figure 2The federated learning system includes a second device and multiple first devices, each of which is deployed with a common feature extraction model and an attribute detection model, and at least some of the first devices are deployed with an individual feature extraction model. The relevant parameters of the attribute detection model and the individual feature extraction model will not be sent to the second device for aggregation, but will only be locally trained by the first device based on its own training sample data. The relevant parameters of the common feature extraction model will be sent to the second device for aggregation, and the aggregated parameters will be issued by the second device to make the common feature extraction models in each of the first devices exactly the same.

[0052] As an example, step S10 includes obtaining a batch of training sample facial images from the first device's private raw data, as well as pre-labeled attribute labels for each training sample. For example, if the first device needs to detect the attribute of whether a mask is worn, the attribute label may be "wearing a mask" or "not wearing a mask"; if the first device needs to detect the attribute of age, the attribute label may be the age of the user in the facial image.

[0053] Step S20, performing local model training on the individual feature extraction model, the common feature extraction model, and the attribute detection model according to the training sample face image and the attribute label, and determining local model update data corresponding to the common feature extraction model;

[0054] In this embodiment, it should be noted that the local model update data refers to the relevant data after the common feature extraction model is updated, which can be the model parameters of the common feature extraction model, such as the weight parameters of the connection between the nodes of the neural network, etc., or it can be the gradient information of the common feature extraction model, such as the gradient information in the neural network gradient descent algorithm.

[0055] As an example, step S20 includes: converting the training sample face image into a training sample face image matrix, inputting the training sample face image matrix into the individual feature extraction model and the common feature extraction model for feature extraction, splicing the features extracted by the individual feature extraction model and the features extracted by the common feature extraction model into a feature vector or feature matrix, inputting them into the attribute detection model, performing attribute detection, and obtaining attribute detection training results, determining the model loss of the attribute detection model based on the difference between the attribute detection training results and their corresponding attribute labels, calculating the gradient based on the model loss, updating the attribute detection model, the individual feature extraction model and the common feature extraction model through back propagation, and obtaining local model update data from the updated common feature extraction model.

[0056] Furthermore, the step of performing local model training on the individual feature extraction model, the common feature extraction model, and the attribute detection model based on the training sample face image and the attribute label, and determining the local model update data corresponding to the common feature extraction model includes:

[0057] Step S21, performing feature extraction on the training sample face image using the individual feature extraction model to obtain individual training sample features, and performing feature extraction on the training sample face image using the common feature extraction model to obtain common training sample features;

[0058] Step S22, inputting the individual training sample features and the common training sample features into the attribute detection model to obtain attribute detection training results;

[0059] Step S23, determining the model loss of the attribute detection model according to the attribute detection training result and the attribute label;

[0060] Step S24: updating the individual feature extraction model and the attribute detection model based on the model loss of the attribute detection model, and determining the local model update data corresponding to the common feature extraction model based on the model loss of the attribute detection model.

[0061] Exemplarily, steps S21-S24 include: converting the training sample face image into a training sample face image matrix, inputting the training sample face image matrix into the individual feature extraction model for feature extraction to obtain at least one individual training sample feature, inputting the training sample face image matrix into the common feature extraction model for feature extraction to obtain at least one common training sample feature, splicing each of the individual training sample features and each of the common training sample features into a feature vector or feature matrix, inputting the feature vector into the attribute detection model, performing attribute detection, and obtaining an attribute detection training result, determining the model loss of the attribute detection model based on the difference between the attribute detection training result and its corresponding attribute label, calculating the gradient based on the model loss, updating the attribute detection model and the individual feature extraction model through back propagation, and determining the local model update data corresponding to the common feature extraction model. The common feature extraction model can be updated based on the local model update data, or it can be temporarily not updated. After federal training, the common feature extraction model is updated based on the federal training result.

[0062] Furthermore, the local model update data includes a second gradient corresponding to the common feature extraction model, and the global model update data includes a global model gradient; the steps of updating the individual feature extraction model and the attribute detection model based on the model loss of the attribute detection model, and determining the local model update data corresponding to the common feature extraction model based on the model loss of the attribute detection model include:

[0063] Step A10, determining a first gradient corresponding to the individual feature extraction model, a second gradient corresponding to the common feature extraction model, and a third gradient corresponding to the attribute detection model according to the model loss of the attribute detection model;

[0064] Step A20: updating the personality feature extraction model based on the first gradient, and updating the attribute detection model based on the third gradient;

[0065] The step of sending the local model update data to the second device so that the second device aggregates the local model update data sent by each of the first devices to obtain global model update data, and sending the global model update data to each of the first devices includes:

[0066] Step A30: Send the second gradient to a second device, so that the second device aggregates the second gradients sent by the first devices to obtain a global model gradient, and sends the global model gradient to the first devices.

[0067] The step of receiving the global model update data sent by the second device and updating the common feature extraction model based on the global model update data includes:

[0068] Step A40: Receive the global model gradient sent by the second device, and update the common feature extraction model based on the global model gradient.

[0069] Exemplarily, steps A10-A40 include: using a gradient descent algorithm to calculate a loss function representing the error between the attribute detection training result and its corresponding attribute label, calculating a third gradient of the loss function relative to each model parameter in the attribute detection model, a first gradient of the loss function relative to each model parameter in the individual feature extraction model, and a second gradient of the loss function relative to each parameter in the common feature extraction model. Subsequently, the individual feature extraction model is updated based on the first gradient, and the attribute detection model is updated based on the third gradient. The common feature extraction model is not updated temporarily. Instead, the second gradient obtained after local training is first determined as the local model update data corresponding to the common feature extraction model and sent to a second device for federated training. After receiving the second gradients sent by each first device, the second device aggregates the second gradients to obtain a global model gradient, which is then sent to each first device. After receiving the global model gradient returned by the second device after federated training, the common feature extraction model is updated based on the global model gradient. Among them, the loss function can adopt a root mean square error loss function, a cross entropy loss function, etc., which can be specifically determined according to the type of attribute detection model, and this embodiment does not limit this.

[0070] Furthermore, the step of updating the individual feature extraction model and the attribute detection model based on the model loss of the attribute detection model, and determining the local model update data corresponding to the common feature extraction model based on the model loss of the attribute detection model includes:

[0071] Step B10: updating the individual feature extraction model, the attribute detection model, and the common feature extraction model based on the model loss of the attribute detection model;

[0072] Step B20: determining the updated model parameters of the common feature extraction model as the local model update data corresponding to the common feature extraction model.

[0073] Exemplarily, steps B10-B20 include: using a gradient descent algorithm to calculate a loss function representing the error between the attribute detection training result and its corresponding attribute label, calculating a third gradient of the loss function relative to each model parameter in the attribute detection model, a first gradient of the loss function relative to each model parameter in the individual feature extraction model, and a second gradient of the loss function relative to each parameter in the common feature extraction model. Furthermore, the individual feature extraction model is updated based on the first gradient, the common feature extraction model is updated based on the second gradient, and the attribute detection model is updated based on the third gradient. Model parameters of the updated common feature extraction model are obtained, determined as local model update data corresponding to the common feature extraction model, and sent to a second device for federated training. After receiving the model parameters sent by each of the first devices, the second device aggregates the model parameters to obtain global model parameters, determines the global model parameters as global model update data, and sends the global model update data to each of the first devices. After receiving the global model update data returned by the second device after federated training, the corresponding model parameters in the common feature extraction model are replaced with the global model update data to update the common feature extraction model. A loss function may be a root mean square error loss function, a cross entropy loss function, or the like, which may be determined based on the type of attribute detection model and is not limited in this embodiment.

[0074] Furthermore, the personality feature extraction model includes an image personality feature extraction model and at least one multimodal personality feature extraction model, and the personality training sample features include image personality training sample features and at least one multimodal personality training sample features;

[0075] The step of extracting features from the training sample face image using the personality feature extraction model to obtain personality training sample features comprises:

[0076] Step S211, obtaining multimodal data of training samples of at least one modality;

[0077] Step S212: According to the mapping relationship between the modality and the multimodal personality feature extraction model, the multimodal data of each training sample is input into the corresponding multimodal personality feature extraction model to obtain at least one multimodal personality training sample feature, and the training sample face image is input into the image personality feature extraction model to obtain the image personality training sample feature.

[0078] When different institutions use different attribute types for facial image detection, the types of sample data obtained may vary due to the different ways in which they collect sample data. For example, some institutions may collect sample data by taking photos, and the sample data obtained is images. Other institutions may collect sample data by shooting videos and recording them, and the sample data obtained may include video and sound. Some institutions may also obtain text information entered by users, such as questionnaires, identity information, etc., and the sample data obtained may include images and text. However, the types of sample data available to different institutions may vary, and therefore the model structures will also vary, making it difficult to train models through horizontal federated learning. Utilizing available multimodal sample data can assist in attribute detection of facial images and effectively improve the accuracy of attribute detection. Especially when training the common feature extraction model corresponding to facial images through federated learning, the accuracy of the common feature extraction model may not reach the most ideal level due to the different requirements of multiple first devices participating in federated learning for detecting different types of attributes. By extracting features from sample data of other modalities that can be obtained, and inputting them into the attribute detection model together with the features of the facial images extracted by the common feature extraction model for attribute detection, the accuracy of attribute detection can be effectively improved.

[0079] In this embodiment, it should be noted that the multimodal data of the training samples refers to sample data collected in modalities other than images; the personality feature extraction model includes an image personality feature extraction model and at least one multimodal personality feature extraction model. The multimodal personality feature extraction model refers to a feature extraction model used to extract features from sample data in modalities other than image data, including at least one of a text personality feature extractor, a video personality feature extractor, and an audio personality feature extractor. The specific method can be determined based on the type of sample data available to each institution, and this embodiment does not impose any restrictions on this. The personality training sample features include image personality training sample features and at least one multimodal personality training sample feature. The multimodal personality training sample features include at least one of a text training sample feature, a video training sample feature, and an audio training sample feature.

[0080] Exemplarily, steps S211-S212 include: obtaining training sample multimodal data corresponding to each of the training samples from the original data private to the first device, converting the training sample multimodal data into a training sample multimodal data matrix, corresponding the training sample multimodal data matrix to each of the multimodal personality feature extraction models one-to-one based on the modality, inputting the training sample multimodal data matrix into the corresponding multimodal personality feature extraction model for feature extraction to obtain at least one multimodal personality training sample feature; and converting the training sample facial image into a training sample facial image matrix, inputting the training sample facial image matrix into the image personality feature extraction model to obtain an image personality training sample feature.

[0081] Step S30: Send the local model update data to the second device, so that the second device aggregates the local model update data sent by each of the first devices to obtain global model update data, and sends the global model update data to each of the first devices;

[0082] As an example, step S30 includes: after the local model is trained, the local model update data corresponding to the updated common feature extraction model is sent to the second device, so that the second device can aggregate the local model update data sent by each of the first devices to obtain global model update data, and send the global model update data to each of the first devices. Among them, the second device can use the FedSGD (Federated Stochastic Gradient Descent) algorithm, FedAvg (Federated Average) algorithm, etc. to aggregate the local model update data. The specific method can be determined according to actual needs, and this embodiment does not limit this.

[0083] Step S40: receive the global model update data sent by the second device, update the common feature extraction model based on the global model update data, and return to execute the step of obtaining the training sample face image and attribute label until the preset federated training end condition is met.

[0084] In this embodiment, it should be noted that the preset federated training end condition may be that the attribute detection models in more than a preset number of first devices among all the first devices participating in the federated learning converge, or that the preset maximum number of iterations of the federated learning is reached, or that the preset maximum training time of the federated learning is reached, etc. The specific conditions can be determined based on actual conditions, and this embodiment does not impose any restrictions on this.

[0085] As an example, step S40 includes: receiving global model update data sent by the second device, updating the common feature extraction model to be the same as the common feature extraction model in each first device participating in federated learning based on the global model update data, returning to execute the step of obtaining the training sample face image and attribute label, starting a new round of federated training until the preset federated training end conditions are met, completing the training of the individual feature extraction model, common feature extraction model and attribute detection model deployed in each first device, and obtaining a trained individual feature extraction model, a trained common feature extraction model and a trained attribute detection model.

[0086] In an implementable manner, without loss of generality, assuming that the current training is in the tth round of federated training, the model of the i-th first device is recorded as It consists of three parts, among which the attribute detection model is recorded as The personality feature extraction model is denoted as The common feature extraction model is denoted as Reference Figure 3 , after local model training, the model becomes Each of the first devices sends the model parameters of the common feature extraction model after local model training to the second device for aggregation, And send the aggregated model parameters to each first device, refer to Figure 4 , each first device updates the common feature extraction model based on the aggregated model parameters.

[0087] In this embodiment, the face attribute detection optimization method is applied to any one of multiple first devices in a federated learning system, and a personality feature extraction model, a common feature extraction model, and an attribute detection model are deployed on the first device. By obtaining training sample face images and attribute labels, local model training is performed on the personality feature extraction model, the common feature extraction model, and the attribute detection model according to the training sample face images and the attribute labels, and the local model update data corresponding to the common feature extraction model is determined, thereby achieving the purpose of local model training of the personality feature extraction model, the common feature extraction model, and the attribute detection model based on the private training sample data of the first device, and then the local model update data is sent to the second device for the second device to perform local model training on each local model update data sent by each first device. Aggregation is performed to obtain global model update data, and the global model update data is sent to each first device, thereby achieving the purpose of jointly training the common feature extraction model by each first device participating in federated learning. More training sample data can be used for training to improve the accuracy of the common feature extraction model, thereby improving the accuracy of common feature extraction in facial images, thereby improving the accuracy of attribute detection of facial images, and then by receiving the global model parameters sent by the second device, the common feature extraction model is updated based on the global model parameters, and the step of obtaining the training sample facial image and attribute label is returned to execute until the preset federated training end condition is met, thereby achieving the training of the individual feature extraction model using the private training sample data of the first device, and the joint training of the common feature extraction model by multiple first devices. When the attributes to be detected by each first device are different, because the sample data used for detection are all face images, even if the attributes to be detected are different or the requirements for model performance are different, the sample features required for detection are often partially the same. Therefore, the common feature extraction model can be used to extract the sample features of the intersection part, and the individual feature extraction model can be used to extract the sample features of the difference part. In this way, compared with the method of training the same feature extraction model for detecting different attributes through federated learning for feature extraction, the present application can realize layered feature extraction and solve the aggregation problem caused by model inconsistency. The common feature extraction model can be trained through federated learning to mine the data commonalities between the first devices and maintain consistency for aggregation, effectively improving the effect of the common feature extraction model. The model structure, training data, etc. of the individual feature extraction model can be flexibly determined according to the own data of each first device and the actual needs of detecting different attributes, focusing on mining the individual feature extraction of local data without being affected by federated learning. Therefore, the effect of the individual feature extraction model will also be better.This overcomes the technical drawback that federated learning requires a unified model structure for all parties, which can lead to poor model performance for some participants and reduced accuracy in facial attribute detection. This improves the accuracy of attribute detection for facial images through federated modeling. Furthermore, only a common feature extraction model needs to be trained on a federated basis, reducing data transmission and improving the training efficiency of federated learning.

[0088] Example 2

[0089] Further, refer to Figure 5 Based on the above embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment can be referred to the above introduction and will not be described in detail. On this basis, the number of the attribute detection models is multiple, each used to detect different attributes, the number of the personality feature extraction models is multiple, and the attribute labels include attribute labels corresponding to each of the attributes;

[0090] The step of performing local model training on the individual feature extraction model, the common feature extraction model, and the attribute detection model based on the training sample face image and the attribute label, and determining the local model update data corresponding to the common feature extraction model comprises:

[0091] Step C10, performing feature extraction on the training sample face images using the individual feature extraction models to obtain a plurality of individual training sample features, and performing feature extraction on the training sample face images using the common feature extraction model to obtain common training sample features;

[0092] In this embodiment, it should be noted that some organizations have their own detection requirements for multiple attributes, for example, they need to detect both age and gender, or they need to detect both age and whether a person is wearing a mask, etc. In this case, if each party participates in federated learning, it will be necessary to repeat a lot of the same work, and the data transmission volume will also double. Therefore, referring to Figure 6 For a first device that needs to detect multiple attributes, only one common feature extraction model can be deployed. Different attribute detections share this common feature extraction model, while different attribute detection models and individual feature extraction models can be deployed for different attributes. This allows multiple groups of models to be trained simultaneously using a single batch of data during local model training, effectively improving model training efficiency. Furthermore, since they all share the same common feature extraction model, data only needs to be transmitted once during federated training, effectively reducing data transmission and improving the training efficiency of federated learning.

[0093] As an example, step C10 includes: converting the training sample face image into a training sample face image matrix, inputting the training sample face image matrix into each of the personality feature extraction models for feature extraction to obtain multiple groups of personality training sample features, each group of personality training sample features may include one or more personality training sample features extracted by the same personality feature extraction model; and inputting the training sample face image matrix into the common feature extraction model for feature extraction to obtain a group of common training sample features, each group of common training sample features may include one or more common training sample features.

[0094] Step C20, based on the mapping relationship between the preset attribute detection model and the personality feature extraction model, combining the characteristics of each personality training sample with the characteristics of the common training sample, and inputting the combined characteristics into the attribute detection model corresponding to each personality training sample characteristic, to obtain attribute detection training results corresponding to different attributes;

[0095] As an example, step C20 includes: determining the attribute detection model corresponding to each of the personality training sample features based on a preset mapping relationship between the attribute detection model and the personality feature extraction model, and based on the personality feature extraction model that outputs each of the personality training sample features. Then, one or more personality training sample features and common training sample features required to be input into the same attribute detection model are concatenated into a feature vector or feature matrix, and the feature vector or feature matrix is ​​input into the corresponding attribute detection model for attribute detection, thereby obtaining attribute detection training results corresponding to different attributes. Each attribute detection model can correspond to one or more personality feature extraction models.

[0096] Step C30, determining the model loss of each attribute detection model according to the attribute detection training results corresponding to each attribute and the attribute labels corresponding to each attribute;

[0097] As an example, step C30 includes: determining the attribute detection training results and the corresponding attribute labels corresponding to each attribute of each training sample, and calculating the loss function that characterizes the error between the attribute detection training results and the corresponding attribute labels corresponding to each attribute of each training sample.

[0098] Step C40: updating each individual feature extraction model, each attribute detection model and / or the common feature extraction model based on the model loss of each attribute detection model, and determining the local model update data corresponding to the common feature extraction model.

[0099] As an example, step C40 includes: calculating the gradient according to the loss function, updating each of the attribute detection models and the individual feature extraction models by back propagation, determining the local model update data corresponding to the common feature extraction model, and the common feature extraction model can be updated based on the local model update data, or it can be temporarily not updated. After federal training, the common feature extraction model is updated according to the federal training results.

[0100] In this embodiment, for the first device with multiple attribute detection requirements, by deploying only one common feature extraction model, different attribute detections share this common feature extraction model, and for different attributes, different attribute detection models and individual feature extraction models are deployed respectively, thereby realizing multi-attribute detection. In this way, when training the local model, multiple groups of models can be trained simultaneously through a batch of data, effectively improving the efficiency of model training, and because they share the same common feature extraction model, they only need to transmit data once when participating in federated training, which can effectively reduce data transmission and improve the training efficiency of federated learning. Different attributes can be detected at the same time. Since they share the same common feature extraction model, the computing resources occupied are reduced, which can effectively improve the overall detection efficiency; detection models of different attributes can be trained at the same time, thereby improving the efficiency of model training.

[0101] Example 3

[0102] Further, refer to Figure 7 Based on the above embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction and will not be described in detail later. On this basis, the facial attribute detection optimization method is applied to a third device, on which a personality feature extraction model, a common feature extraction model, and an attribute detection model are deployed, wherein the personality feature extraction model, the common feature extraction model, and the attribute detection model are trained using the facial attribute detection optimization method described above; the facial attribute detection optimization method includes the following steps:

[0103] Step D10, obtaining a face image to be detected;

[0104] In this embodiment, it should be noted that the facial attribute detection optimization method is applied to a third device, which can be the first device. After completing model training for the individual feature extraction model, the common feature extraction model, and the attribute detection model, attribute detection can be performed directly locally. The third device can also be another electronic device other than the first device. After completing model training for the individual feature extraction model, the common feature extraction model, and the attribute detection model using the first device, the trained individual feature extraction model, the common feature extraction model, and the attribute detection model are deployed on the other electronic device for attribute detection. The specific method can be determined based on actual conditions and is not limited in this embodiment.

[0105] As an example, step D10 includes: obtaining a face image to be detected of a sample to be detected.

[0106] Step D20, performing feature extraction on the face image to be detected using the individual feature extraction model to obtain individual detection features, and performing feature extraction on the face image to be detected using the common feature extraction model to obtain common detection features;

[0107] As an example, step D20 includes: converting the facial image to be detected into a facial image matrix to be detected, inputting the facial image matrix to be detected into the individual feature extraction model for feature extraction to obtain at least one individual detection feature, and inputting the facial image matrix to be detected into the common feature extraction model for feature extraction to obtain at least one common detection feature.

[0108] Step D30: Input the individuality detection feature and the commonality detection feature into an attribute detection model to obtain an attribute detection result.

[0109] As an example, the step D30 includes: splicing each of the individual detection features and each of the common detection features into a feature vector or a feature matrix, inputting the feature vector into the attribute detection model, performing attribute detection, and obtaining an attribute detection result.

[0110] In this embodiment, facial image features are divided into common features and individual features, which are extracted separately. Since the common feature extraction model is trained through federated learning, the number of samples used for model training is larger and richer, thereby improving the accuracy of common feature extraction. Furthermore, since common feature extraction and individual feature extraction are separated, the common feature extraction model is not affected by the different individual features of different attributes during federated training. Furthermore, since an individual feature extraction model is also deployed, the specific individual sample features required for different attribute detection are not lost due to federated learning. In this way, both more accurate common sample features and the individual sample features required for local attribute detection can be extracted. The combination of the two can effectively improve the accuracy of attribute detection.

[0111] Example 4

[0112] Furthermore, the embodiment of the present application also provides a face attribute detection optimization device, referring to Figure 8 The facial attribute detection and optimization device is applied to any one of multiple first devices in the federated learning system, where an individual feature extraction model, a common feature extraction model, and an attribute detection model are deployed on the first device. The facial attribute detection and optimization device includes:

[0113] An acquisition module 10 is used to acquire training sample face images and attribute labels;

[0114] A local training module 20 is configured to perform local model training on the individual feature extraction model, the common feature extraction model, and the attribute detection model based on the training sample face image and the attribute label, and determine local model update data corresponding to the common feature extraction model;

[0115] a sending module 30, configured to send the local model update data to a second device, so that the second device aggregates the local model update data sent by each of the first devices to obtain global model update data, and sends the global model update data to each of the first devices;

[0116] The update module 40 is used to receive the global model update data sent by the second device, update the common feature extraction model based on the global model update data, and return to execute the step of obtaining the training sample face image and attribute label until the preset federal training end condition is met.

[0117] Optionally, the local training module 20 is further configured to:

[0118] Performing feature extraction on the training sample face images using the individual feature extraction model to obtain individual training sample features, and performing feature extraction on the training sample face images using the common feature extraction model to obtain common training sample features;

[0119] Inputting the individual training sample features and the common training sample features into the attribute detection model to obtain attribute detection training results;

[0120] Determining a model loss of the attribute detection model according to the attribute detection training result and the attribute label;

[0121] The individual feature extraction model and the attribute detection model are updated based on the model loss of the attribute detection model, and local model update data corresponding to the common feature extraction model is determined based on the model loss of the attribute detection model.

[0122] Optionally, the local model update data includes a second gradient corresponding to the common feature extraction model, the global model update data includes a global model gradient, and the local training module 20 is further configured to:

[0123] Determine, according to the model loss of the attribute detection model, a first gradient corresponding to the individual feature extraction model, a second gradient corresponding to the common feature extraction model, and a third gradient corresponding to the attribute detection model;

[0124] updating the personality feature extraction model based on the first gradient, and updating the attribute detection model based on the third gradient;

[0125] The sending module 30 is further configured to:

[0126] Sending the second gradient to a second device, so that the second device aggregates the second gradients sent by the first devices to obtain a global model gradient, and sending the global model gradient to the first devices;

[0127] The updating module 40 is further configured to:

[0128] Receive the global model gradient sent by the second device, and update the common feature extraction model based on the global model gradient.

[0129] Optionally, the local training module 20 is further configured to:

[0130] Updating the individual feature extraction model, the attribute detection model, and the common feature extraction model based on the model loss of the attribute detection model;

[0131] The updated model parameters of the common feature extraction model are determined as the local model update data corresponding to the common feature extraction model.

[0132] Optionally, the personality feature extraction model includes an image personality feature extraction model and at least one multimodal personality feature extraction model, the personality training sample features include image personality training sample features and at least one multimodal personality training sample features, and the local training module 20 is further configured to:

[0133] Obtaining multimodal data of training samples of at least one modality;

[0134] According to the mapping relationship between the modality and the multimodal personality feature extraction model, the multimodal data of each training sample is input into the corresponding multimodal personality feature extraction model to obtain at least one multimodal personality training sample feature, and the training sample face image is input into the image personality feature extraction model to obtain the image personality training sample feature.

[0135] Optionally, there are multiple attribute detection models, each for detecting a different attribute; there are multiple personality feature extraction models; the attribute labels include attribute labels corresponding to each of the attributes; and the local training module 20 is further configured to:

[0136] Performing feature extraction on the training sample face images using the individual feature extraction models to obtain a plurality of individual training sample features, and performing feature extraction on the training sample face images using the common feature extraction model to obtain common training sample features;

[0137] According to the mapping relationship between the preset attribute detection model and the personality feature extraction model, the characteristics of each personality training sample are combined with the characteristics of the common training sample, and then input into the attribute detection model corresponding to each personality training sample characteristic, to obtain the attribute detection training results corresponding to different attributes;

[0138] Determining the model loss of each attribute detection model according to the attribute detection training results corresponding to each of the attributes and the attribute labels corresponding to each of the attributes;

[0139] Based on the model loss of each attribute detection model, each individual feature extraction model, each attribute detection model and / or the common feature extraction model is updated, and local model update data corresponding to the common feature extraction model is determined.

[0140] The facial attribute detection optimization device provided by the present invention utilizes the facial attribute detection optimization method described in the aforementioned embodiments, resolving the technical issue of low accuracy in attribute detection of facial images using federated modeling in the prior art. Compared to the prior art, the benefits of the facial attribute detection optimization device provided by the present invention are similar to those of the facial attribute detection optimization method described in the aforementioned embodiments. Other technical features of the facial attribute detection optimization device are the same as those disclosed in the aforementioned embodiments and are not further elaborated here.

[0141] Example 5

[0142] Furthermore, an embodiment of the present application further provides a facial attribute detection optimization device, which is applied to a third device. An individual feature extraction model, a common feature extraction model, and an attribute detection model are deployed on the third device. The individual feature extraction model, the common feature extraction model, and the attribute detection model are trained using the facial attribute detection optimization method described above. The facial attribute detection optimization device includes:

[0143] An acquisition module is used to acquire the face image to be detected;

[0144] A feature extraction module is used to extract features from the face image to be detected using an individual feature extraction model to obtain individual detection features, and to extract features from the face image to be detected using a common feature extraction model to obtain common detection features;

[0145] The detection module is used to input the individual detection feature and the common detection feature into an attribute detection model to obtain an attribute detection result.

[0146] Example 6

[0147] Furthermore, an embodiment of the present invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the face attribute detection optimization method in the above embodiment.

[0148] Reference below Figure 9 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as Bluetooth headsets, mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0149] like Figure 9 As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and arrays required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0150] Typically, the following systems can be connected to the I / O interface: input devices including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices including, for example, magnetic tape, hard disk, etc.; and communication devices. The communication devices can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figures show electronic devices with various systems, it should be understood that not all of the illustrated systems are required to be implemented or present. More or fewer systems may be implemented or present instead.

[0151] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0152] The electronic device provided by the present invention utilizes the facial attribute detection optimization method described in the above-mentioned embodiments, resolving the technical issue of low accuracy in attribute detection of facial images using federated modeling in the prior art. Compared to the prior art, the electronic device provided by the present invention provides the same benefits as the facial attribute detection optimization method described in the above-mentioned embodiments. Other technical features of the electronic device are the same as those disclosed in the above-mentioned embodiments and are not further detailed here.

[0153] It should be understood that various parts of the present disclosure can be implemented with hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in an appropriate manner.

[0154] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0155] Example 7

[0156] Furthermore, this embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the face attribute detection optimization method in the above embodiment.

[0157] The computer-readable storage medium provided in the embodiment of the present invention can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0158] The computer-readable storage medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0159] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the electronic device: obtains training sample facial images and attribute labels; performs local model training on the individual feature extraction model, the common feature extraction model and the attribute detection model according to the training sample facial images and the attribute labels, and determines the local model update data corresponding to the common feature extraction model; sends the local model update data to a second device, so that the second device can aggregate the local model update data sent by each first device to obtain global model update data, and send the global model update data to each first device; receives the global model update data sent by the second device, updates the common feature extraction model based on the global model update data, and returns to execute the step of obtaining training sample facial images and attribute labels until the preset federal training end condition is met.

[0160] Alternatively, the computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device: obtains a facial image to be detected; performs feature extraction on the facial image to be detected through a personality feature extraction model to obtain personality detection features, and performs feature extraction on the facial image to be detected through a common feature extraction model to obtain common detection features; inputs the personality detection features and the common detection features into an attribute detection model to obtain an attribute detection result.

[0161] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0162] The flow charts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0163] The modules involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0164] The computer-readable storage medium provided by the present invention stores computer-readable program instructions for executing the aforementioned facial attribute detection optimization method, resolving the technical issue of low accuracy in attribute detection of facial images using federated modeling in the prior art. Compared to the prior art, the benefits of the computer-readable storage medium provided by the present invention are similar to those of the facial attribute detection optimization method provided by the aforementioned embodiment and are not further elaborated here.

[0165] Example 8

[0166] Furthermore, the present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned facial attribute detection optimization method when executed by a processor.

[0167] The computer program product provided in this application solves the technical problem of low accuracy in attribute detection of facial images using federated modeling in the prior art. Compared to the prior art, the benefits of the computer program product provided in this embodiment of the present invention are similar to those of the facial attribute detection optimization method provided in the aforementioned embodiment, and are not further elaborated here.

[0168] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A facial attribute detection optimization method, characterized in that: The facial attribute detection optimization method is applied to any one of multiple first devices in a federated learning system, where an individual feature extraction model, a common feature extraction model, and an attribute detection model are deployed on the first device; the facial attribute detection optimization method comprises the following steps: Obtain training sample face images and attribute labels; Performing local model training on the individual feature extraction model, the common feature extraction model, and the attribute detection model according to the training sample face image and the attribute label, and determining local model update data corresponding to the common feature extraction model; Sending the local model update data to a second device, so that the second device aggregates the local model update data sent by each of the first devices to obtain global model update data, and sending the global model update data to each of the first devices; Receiving global model update data sent by the second device, updating the common feature extraction model based on the global model update data, and returning to the step of obtaining training sample facial images and attribute labels until a preset federated training end condition is met; The step of performing local model training on the individual feature extraction model, the common feature extraction model, and the attribute detection model based on the training sample face image and the attribute label, and determining the local model update data corresponding to the common feature extraction model comprises: Performing feature extraction on the training sample face images using the individual feature extraction model to obtain individual training sample features, and performing feature extraction on the training sample face images using the common feature extraction model to obtain common training sample features; Inputting the individual training sample features and the common training sample features into the attribute detection model to obtain attribute detection training results; Determining a model loss of the attribute detection model according to the attribute detection training result and the attribute label; updating the individual feature extraction model and the attribute detection model based on the model loss of the attribute detection model, and determining local model update data corresponding to the common feature extraction model based on the model loss of the attribute detection model; The local model update data includes a second gradient corresponding to the common feature extraction model, and the steps of updating the individual feature extraction model and the attribute detection model based on the model loss of the attribute detection model, and determining the local model update data corresponding to the common feature extraction model based on the model loss of the attribute detection model include: Determine, according to the model loss of the attribute detection model, a first gradient corresponding to the individual feature extraction model, a second gradient corresponding to the common feature extraction model, and a third gradient corresponding to the attribute detection model; updating the personality feature extraction model based on the first gradient, and updating the attribute detection model based on the third gradient; Alternatively, the step of updating the individual feature extraction model and the attribute detection model based on the model loss of the attribute detection model, and determining the local model update data corresponding to the common feature extraction model based on the model loss of the attribute detection model includes: Updating the individual feature extraction model, the attribute detection model, and the common feature extraction model based on the model loss of the attribute detection model; The updated model parameters of the common feature extraction model are determined as the local model update data corresponding to the common feature extraction model.

2. The face attribute detection optimization method according to claim 1, wherein: The global model update data includes a global model gradient; the step of sending the local model update data to a second device so that the second device aggregates the local model update data sent by each of the first devices to obtain global model update data, and sending the global model update data to each of the first devices includes: Sending the second gradient to a second device, so that the second device aggregates the second gradients sent by the first devices to obtain a global model gradient, and sending the global model gradient to the first devices; The step of receiving the global model update data sent by the second device and updating the common feature extraction model based on the global model update data includes: Receive the global model gradient sent by the second device, and update the common feature extraction model based on the global model gradient.

3. The face attribute detection optimization method according to claim 1, wherein: The personality feature extraction model includes an image personality feature extraction model and at least one multimodal personality feature extraction model, and the personality training sample features include an image personality training sample feature and at least one multimodal personality training sample feature; The step of extracting features from the training sample face image using the personality feature extraction model to obtain personality training sample features comprises: Obtaining multimodal data of training samples of at least one modality; According to the mapping relationship between the modality and the multimodal personality feature extraction model, the multimodal data of each training sample is input into the corresponding multimodal personality feature extraction model to obtain at least one multimodal personality training sample feature, and the training sample face image is input into the image personality feature extraction model to obtain the image personality training sample feature.

4. The method for optimizing facial attribute detection according to claim 1, wherein: There are multiple attribute detection models, each used to detect a different attribute. There are multiple personality feature extraction models, and the attribute labels include attribute labels corresponding to each of the attributes. The step of performing local model training on the individual feature extraction model, the common feature extraction model, and the attribute detection model based on the training sample face image and the attribute label, and determining the local model update data corresponding to the common feature extraction model comprises: Performing feature extraction on the training sample face images using the individual feature extraction models to obtain a plurality of individual training sample features, and performing feature extraction on the training sample face images using the common feature extraction model to obtain common training sample features; According to the mapping relationship between the preset attribute detection model and the personality feature extraction model, the characteristics of each personality training sample are combined with the characteristics of the common training sample, and then input into the attribute detection model corresponding to each personality training sample characteristic, to obtain the attribute detection training results corresponding to different attributes; Determining the model loss of each attribute detection model according to the attribute detection training results corresponding to each of the attributes and the attribute labels corresponding to each of the attributes; Based on the model loss of each attribute detection model, each individual feature extraction model, each attribute detection model and / or the common feature extraction model is updated, and local model update data corresponding to the common feature extraction model is determined.

5. A facial attribute detection optimization method, characterized in that: The facial attribute detection optimization method is applied to a third device, wherein a personality feature extraction model, a common feature extraction model, and an attribute detection model are deployed on the third device, wherein the personality feature extraction model, the common feature extraction model, and the attribute detection model are trained using the facial attribute detection optimization method according to any one of claims 1 to 4; the facial attribute detection optimization method comprises the following steps: Get the face image to be detected; Performing feature extraction on the face image to be detected by using an individual feature extraction model to obtain individual detection features, and performing feature extraction on the face image to be detected by using a common feature extraction model to obtain common detection features; The individuality detection feature and the commonality detection feature are input into an attribute detection model to obtain an attribute detection result.

6. A facial attribute detection and optimization device, applied to any one of multiple first devices in a federated learning system, wherein an individual feature extraction model, a common feature extraction model, and an attribute detection model are deployed on the first device, and the facial attribute detection and optimization device comprises: Acquisition module, used to obtain training sample face images and attribute labels; A local training module is used to perform local model training on the individual feature extraction model, the common feature extraction model, and the attribute detection model based on the training sample facial images and the attribute labels, and determine the local model update data corresponding to the common feature extraction model; the local training module is also used to perform feature extraction on the training sample facial images using the individual feature extraction model to obtain individual training sample features, and to perform feature extraction on the training sample facial images using the common feature extraction model to obtain common training sample features; the individual training sample features and the common training sample features are input into the attribute detection model to obtain attribute detection training results; and the model loss of the attribute detection model is determined based on the attribute detection training results and the attribute labels; updating the individual feature extraction model and the attribute detection model based on the model loss of the attribute detection model, and determining local model update data corresponding to the common feature extraction model based on the model loss of the attribute detection model; The local model update data includes a second gradient corresponding to the common feature extraction model, and the local training module is further configured to determine, based on the model loss of the attribute detection model, a first gradient corresponding to the individual feature extraction model, a second gradient corresponding to the common feature extraction model, and a third gradient corresponding to the attribute detection model; update the individual feature extraction model based on the first gradient, and update the attribute detection model based on the third gradient; Or the local training module is further used to update the individual feature extraction model, the attribute detection model and the common feature extraction model based on the model loss of the attribute detection model; Determining the updated model parameters of the common feature extraction model as local model update data corresponding to the common feature extraction model; a sending module, configured to send the local model update data to a second device, so that the second device aggregates the local model update data sent by each of the first devices to obtain global model update data, and sends the global model update data to each of the first devices; An update module is used to receive the global model update data sent by the second device, update the common feature extraction model based on the global model update data, and return to execute the step of obtaining the training sample face image and attribute label until the preset federated training end condition is met.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the facial attribute detection optimization method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which is stored a program for implementing the facial attribute detection optimization method. The program for implementing the facial attribute detection optimization method is executed by a processor to implement the steps of the facial attribute detection optimization method as described in any one of claims 1 to 5.

9. A product, said product being a computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the face attribute detection optimization method according to any one of claims 1 to 5 are implemented.

Images