Face analysis method, electronic equipment and storage medium

Abstract

The invention discloses a face analysis method, electronic equipment and a storage medium, and relates to the technical field of face attribute recognition, and the face analysis method comprises the steps: inputting a face image into a multi-scale feature coding module, and obtaining an image feature code; inputting the image feature code, a preset task information code and a preset image position code into a decoding attention unit to obtain a task related feature and an updated image feature code; the task related features are decomposed into subtask related features through a decoding output unit, and face analysis results corresponding to the subtasks are obtained according to the subtask related features and updated image feature coding. According to the method, the analysis accuracy and the system integration efficiency of the face analysis system can be improved, so that the overall performance of the face analysis system is improved.

Description

Technical Field

[0001] The present application relates to the technical field of face attribute recognition, and in particular to a face analysis method, an electronic device and a storage medium. Background Art

[0002] As a cutting-edge research direction in the field of computer vision, face analysis is widely used in many fields such as security monitoring and personalized services.

[0003] The current face analysis technology framework includes multiple subtasks such as face parsing, key point location, posture estimation, face recognition, age, gender, race estimation and key point visibility prediction. Traditional face analysis designs independent network structure and preprocessing process for each subtask, but this approach will lead to information isolation and error accumulation, affecting the accuracy and stability of the overall analysis. At the same time, the independent development of modules increases the complexity of system integration, resulting in low system integration efficiency.

[0004] In summary, how to improve the overall performance of the face analysis system has become a technical problem that needs to be urgently solved in this field. Summary of the invention

[0005] The main purpose of this application is to provide a face analysis method, an electronic device and a storage medium, aiming to improve the overall performance of a face analysis system by improving the analysis accuracy and system integration efficiency of the face analysis system.

[0006] To achieve the above object, the present application proposes a face analysis method, which performs face analysis based on a face analysis model, wherein the face analysis model includes a multi-scale feature encoding module and a task decoding module, wherein the task decoding module includes a decoding attention unit and a decoding output unit, and the face analysis method includes:

[0007] Inputting the face image into the multi-scale feature encoding module to obtain image feature encoding;

[0008] Inputting the image feature code, the preset task information code and the preset image position code into the decoding attention unit to obtain task-related features and update the image feature code;

[0009] The task-related features are decomposed into sub-task-related features by the decoding output unit, and the face analysis results corresponding to each sub-task are obtained according to the sub-task-related features and the updated image feature encoding.

[0010] In one embodiment, the multi-scale feature encoding module includes a multi-scale facial feature acquisition unit and a feature fusion unit, and the step of inputting the facial image into the multi-scale feature encoding module to obtain the image feature encoding includes:

[0011] Inputting the facial image into the multi-scale facial feature acquisition unit to obtain facial features of multiple different scales;

[0012] Each of the facial features is input into the feature fusion unit for feature fusion processing to obtain image feature encoding with unified scale.

[0013] In one embodiment, the task decoding module further includes an image position encoding unit and a task information encoding unit, and before the steps of inputting the image feature code, the preset task information code and the preset image position code into the decoding attention unit, obtaining task-related features and updating the image feature code, further includes:

[0014] Generate an image position code by the image position coding unit;

[0015] Through the task information encoding unit, each subtask is initialized as a learnable feature vector, and each feature vector is concatenated to obtain the task information encoding.

[0016] In one embodiment, the decoding attention unit includes a bidirectional transformer block and a cross attention sub-unit, and the step of inputting the image feature code, the preset task information code and the preset image position code into the decoding attention unit to obtain task-related features and update the image feature code includes:

[0017] Input the image feature code, the preset task information code and the preset image position code into the bidirectional transformer (a neural network architecture based on the attention mechanism) block for self-attention mechanism and cross-attention mechanism processing to obtain intermediate task-related features and intermediate image feature codes;

[0018] The intermediate task-related features and the intermediate image feature codes are input into the cross-attention subunit for cross-attention mechanism processing to obtain task-related features and updated image feature codes.

[0019] In one embodiment, the step of inputting the image feature code, the preset task information code and the preset image position code into the bidirectional transformer block for self-attention mechanism and cross-attention mechanism processing to obtain intermediate task-related features and intermediate image feature codes includes:

[0020] The preset task information encoding is processed by the self-attention mechanism through the bidirectional transformer block to obtain a new task information encoding;

[0021] The new task information encoding, the image feature encoding and the preset image position encoding are processed by a cross-attention mechanism to obtain intermediate task-related features and intermediate image feature encodings.

[0022] In one embodiment, the subtasks include facial key point prediction, facial posture prediction, facial age prediction, facial gender prediction, and facial race prediction, and the step of decomposing the task-related features into subtask-related features through the decoding output unit, and obtaining the facial analysis results corresponding to each subtask according to the subtask-related features and the updated image feature encoding includes:

[0023] Decomposing the task-related features into sub-task-related features through the decoding output unit, wherein the sub-task-related features include facial key point features, facial posture features, facial age features, facial gender features, and facial ethnicity features;

[0024] After the facial key point features, the facial posture features, the facial age features, the facial gender features and the facial race features are respectively subjected to multi-layer MLP (Multi-Layer Perceptron) feature conversion processing, the facial analysis results corresponding to the facial key point prediction, the facial posture prediction, the facial age prediction, the facial gender prediction and the facial race prediction are obtained.

[0025] In one embodiment, the task decoding module further includes an upsampling unit, the subtask further includes face semantic segmentation prediction, and the step of decomposing the task-related features into subtask-related features through the decoding output unit, and obtaining the face analysis results corresponding to each subtask according to the subtask-related features and the updated image feature encoding includes:

[0026] Decomposing the task-related features into sub-task-related features through the decoding output unit, wherein the sub-task-related features include face semantic segmentation features;

[0027] The face semantic segmentation feature is subjected to multi-layer MLP layer feature conversion processing to obtain an updated face semantic segmentation feature;

[0028] Performing an upsampling operation on the updated image feature code by the upsampling unit to obtain an updated image feature code matrix;

[0029] The updated face semantic segmentation feature is multiplied by the updated image feature encoding matrix to obtain a face analysis result corresponding to the face semantic segmentation prediction.

[0030] In one embodiment, before inputting the face image into the multi-scale feature encoding module to obtain the image feature encoding, the process further includes:

[0031] Acquire a training sample, wherein the training sample includes a plurality of labeled training face images;

[0032] Inputting the training face image into an initial face analysis model to obtain a training face analysis result;

[0033] Calculating a joint loss function according to the training face analysis result and the true similarity value of the training sample, wherein the joint loss function is obtained by weighted summation of a face key point prediction task loss function, a face posture prediction task loss function, a face age prediction task loss function, a face gender prediction task loss function, a face race prediction task loss function, and a face semantic segmentation task loss function;

[0034] The parameters of the initial face analysis model are updated according to the joint loss function to obtain a trained face analysis model.

[0035] In one embodiment, before the step of calculating the joint loss function according to the training face analysis result and the true similarity value of the training sample, the method further includes:

[0036] Constructing the face key point prediction task loss function based on the Wingloss loss function;

[0037] Constructing the face pose prediction task loss function based on the rotation matrix geodesic distance loss function;

[0038] Based on the cross entropy loss function, the face age prediction task loss function, the face gender prediction task loss function and the face race prediction task loss function are respectively constructed;

[0039] The face semantic segmentation task loss function is constructed based on the second cross entropy loss function.

[0040] In addition, to achieve the above purpose, the present application also proposes a face analysis system, which performs face analysis based on a face analysis model, wherein the face analysis model includes a multi-scale feature encoding module and a task decoding module, wherein the task decoding module includes a decoding attention unit and a decoding output unit, and the face analysis system includes:

[0041] A feature acquisition module, used for inputting a face image into the multi-scale feature encoding module to obtain an image feature code;

[0042] A feature updating module, used for inputting the image feature code, the preset task information code and the preset image position code into the decoding attention unit to obtain task-related features and update the image feature code;

[0043] The face analysis module is used to decompose the task-related features into sub-task-related features through the decoding output unit, and obtain the face analysis results corresponding to each sub-task according to the sub-task-related features and the updated image feature encoding.

[0044] In addition, to achieve the above-mentioned purpose, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the face analysis method as described above.

[0045] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the face analysis method as described above are implemented.

[0046] The present application proposes a face analysis method. The present application performs face analysis based on a face analysis model. The face analysis model includes a multi-scale feature encoding module and a task decoding module. The task decoding module includes a decoding attention unit and a decoding output unit. The face image to be analyzed is input into the multi-scale feature encoding module. The multi-scale feature encoding module can extract the image feature code in the face image. Then, the extracted image feature code, the preset task information code (used to indicate each face analysis subtask that needs to be executed currently) and the image position code (used to mark the position information of the feature in the image) are input into the decoding attention unit. In the decoding attention unit, the task-related features related to the face analysis task are obtained through the attention mechanism, and the image feature code is updated at the same time to make it more focused on the task requirements. Finally, the task-related features are decomposed into each sub-task-related features through the decoding output unit, and the face analysis results corresponding to each sub-task are obtained according to the sub-task-related features and the updated image feature code.

[0047] In summary, this application integrates multiple face analysis subtasks into a unified framework by adopting a face analysis model, thereby realizing effective information sharing and interaction and simplifying the complexity of system integration; at the same time, through the collaborative work of the decoding attention unit and the decoding output unit, it can flexibly process and output according to the needs of different subtasks, thereby improving the accuracy and stability of the overall analysis, thereby improving the overall performance of the face analysis system. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 A schematic diagram of a flow chart provided for the first embodiment of the face analysis method of the present application;

[0051] Figure 2 A schematic diagram of the structure of a multi-scale feature encoding module provided in Example 2 of the face analysis method of the present application;

[0052] Figure 3 A schematic diagram of the decoding attention unit structure provided in Example 2 of the face analysis method of the present application;

[0053] Figure 4 A schematic diagram of a bidirectional transformer block structure provided in Example 2 of the face analysis method of the present application;

[0054] Figure 5 A schematic diagram of the structure of a task decoding module provided in Example 2 of the face analysis method of the present application;

[0055] Figure 6 This is a schematic diagram of the module structure of the face analysis system according to an embodiment of the present application;

[0056] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the face analysis method in the embodiment of the present application.

[0057] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0059] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0060] The main solution of the embodiment of the present application is: inputting the face image into the multi-scale feature coding module to obtain the image feature code; inputting the image feature code, the preset task information code and the preset image position code into the decoding attention unit to obtain the task-related features and the updated image feature code; decomposing the task-related features into sub-task-related features through the decoding output unit, and obtaining the face analysis results corresponding to each sub-task according to the sub-task-related features and the updated image feature code.

[0061] The current face analysis technology framework includes multiple subtasks such as face parsing, key point location, posture estimation, face recognition, age, gender, race estimation and key point visibility prediction. Traditional face analysis designs independent network structure and preprocessing process for each subtask, but this approach will lead to information isolation and error accumulation, affecting the accuracy and stability of the overall analysis. At the same time, the independent development of modules increases the complexity of system integration, resulting in low system integration efficiency.

[0062] The embodiment of the present application provides a solution that integrates multiple face analysis subtasks into a unified framework by adopting a face analysis model, thereby achieving effective information sharing and interaction and simplifying the complexity of system integration; at the same time, through the collaborative work of the decoding attention unit and the decoding output unit, it can flexibly process and output according to the needs of different subtasks, thereby improving the accuracy and stability of the overall analysis, thereby improving the overall performance of the face analysis system.

[0063] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a server, a computer, etc., or an electronic device capable of realizing the above functions, etc. The following takes a computer terminal as an example to illustrate this embodiment and the following embodiments.

[0064] Based on this, the present application embodiment provides a face analysis method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the face analysis method of the present application.

[0065] In this embodiment, the face analysis method performs face analysis based on a face analysis model. The face analysis model includes a multi-scale feature encoding module and a task decoding module. The task decoding module includes a decoding attention unit and a decoding output unit. The face analysis method includes steps S10 to S30:

[0066] Step S10, inputting the face image into a multi-scale feature encoding module to obtain image feature encoding;

[0067] The face image to be analyzed is input into the multi-scale feature coding module, and the image feature coding is extracted from the face image through the multi-scale feature coding module.

[0068] In a feasible implementation, the multi-scale feature encoding module can adopt the hierarchical construction method in CNN (Convolutional Neural Networks), such as Resnet (Residual Network), Mobilenet (a lightweight convolutional neural network) or Swin transformer (a new deep learning architecture for computer vision tasks), etc. In the embodiment of the present application, considering the original intention of lightweight design, the MobilenetV3 (a lightweight deep learning model) structure is used to construct the multi-scale feature encoding module. Of course, other lightweight architectures can also be used, and this is not specifically limited in this embodiment.

[0069] In addition, the face image needs to be preprocessed before entering the multi-scale feature encoding module. Use an open source face detection algorithm (such as the FaceBox face detection algorithm) to detect and locate the face in the image, return the face frame coordinates, and use the face frame as the center to expand 25% up, down, left, and right to get the face image. Then adjust the face image size to a fixed 256*256 when entering the multi-scale feature encoding module.

[0070] Step S20, inputting the image feature code, the preset task information code and the preset image position code into the decoding attention unit to obtain the task-related features and update the image feature code;

[0071] The extracted image feature code, the preset task information code (used to indicate the various face analysis subtasks that need to be executed currently) and the image position code (used to mark the location information of the feature in the image) are input into the decoding attention unit together. In the decoding attention unit, the task-related features related to the face analysis task are obtained through the attention mechanism, and the image feature code is updated at the same time to make it more focused on the task requirements.

[0072] Step S30, decomposing the task-related features into sub-task-related features through a decoding output unit, and obtaining face analysis results corresponding to each sub-task according to the sub-task-related features and the updated image feature encoding.

[0073] The task-related features are passed to the decoding output unit, and the decoding output unit uses its internal decoding algorithm and mechanism to decompose the task-related features into sub-task-related features. Each sub-task-related feature corresponds to a specific sub-task in face analysis, such as precise positioning of facial key points, accurate identification of facial attributes, etc. Then, the face analysis results corresponding to each sub-task are calculated based on these sub-task-related features and the updated image feature coding.

[0074] In this way, this embodiment integrates multiple face analysis subtasks into a unified framework by adopting a face analysis model, thereby realizing effective information sharing and interaction and simplifying the complexity of system integration; at the same time, through the collaborative work of the decoding attention unit and the decoding output unit, it can flexibly process and output according to the needs of different subtasks, thereby improving the accuracy and stability of the overall analysis, thereby improving the overall performance of the face analysis system.

[0075] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and no further description will be given later. On this basis, the multi-scale feature encoding module includes a multi-scale face feature acquisition unit and a feature fusion unit, and step S10 may include steps S101 to S102:

[0076] Step S101, inputting a face image into a multi-scale face feature acquisition unit to obtain face features of multiple different scales;

[0077] The face image to be analyzed is input into a multi-scale face feature acquisition unit, which can acquire face features of the face image at different scales, each face feature including unique information of the face in the face image at different scales.

[0078] Step S102: input each facial feature into a feature fusion unit for feature fusion processing to obtain image feature coding with a unified scale.

[0079] The obtained facial features are input into the feature fusion unit, which can fuse these facial features from different scales to generate an image feature code with uniform scale and rich information.

[0080] For example, in a feasible implementation manner, the structural diagram of the multi-scale feature encoding module is as follows: Figure 2As shown, the multi-scale feature encoding module is used to extract and encode features of different scales to improve the recognition ability of the model. The multi-scale feature encoding module includes a multi-scale face feature acquisition unit and a feature fusion unit, wherein the Stem (the initial part of the model) in the multi-scale face feature acquisition unit is used to preliminarily process the input face image and extract the preliminary features in the face image, which may include a convolution layer, a normalization layer, and an activation function, etc. Module 1, module 2, module 3, and module 4 are continuous feature extraction blocks in the multi-scale face feature acquisition unit, each block contains multiple convolution layers, pooling layers, or other types of layers, which are used to gradually extract deeper features. The size of the output feature map of each module will gradually decrease to capture richer feature information; the interpolation processing in the feature fusion unit refers to bilinear interpolation of the face features output by each feature extraction block to unify the size of each face feature. It is worth mentioning that in the process of fusion of face features of different scales, only the features of modules 2 to 4 are selected for fusion, in order to reduce the memory of the deployment. Splicing and convolution refer to splicing the interpolated facial features and then fusing them through the convolution layer to integrate facial features of different scales together to form a comprehensive image feature encoding, which helps the model recognize faces at different scales.

[0081] In this way, the design of the entire multi-scale feature encoding module allows the face analysis model to capture and encode features at multiple scales, which can improve the accuracy and robustness of the face analysis model in performing face analysis.

[0082] In a feasible embodiment, the task decoding module further includes an image position encoding unit and a task information encoding unit, and steps S40 to S50 may be included before step S20:

[0083] Step S40, generating an image position code by an image position coding unit;

[0084] The image position coding unit generates a series of accurate image position codes according to the specific content and structure of the input face image, and the image position codes can reflect the position information of each part in the face image.

[0085] Step S50, through the task information encoding unit, each subtask is initialized as a learnable feature vector, and each feature vector is concatenated to obtain the task information code.

[0086] Through the task information encoding unit, each subtask (such as facial key point prediction, facial posture prediction, etc.) is initialized as a learnable feature vector. The feature vector can be continuously optimized and updated through learning to adapt to different face analysis tasks. Then, the task information encoding unit splices the feature vectors of each subtask to generate a task information code containing all subtask information.

[0087] In a feasible embodiment, the decoding attention unit includes a bidirectional transformer block and a cross attention sub-unit, and step S20 may include steps S201-S202:

[0088] Step S201, inputting the image feature code, the preset task information code and the preset image position code into the bidirectional transformer block for self-attention mechanism and cross-attention mechanism processing to obtain intermediate task-related features and intermediate image feature codes;

[0089] The image feature encoding, task information encoding and image position encoding are sent to the bidirectional transformer block, which uses the self-attention mechanism and the cross-attention mechanism to deeply process the input data. The self-attention mechanism allows the model to consider the information of other elements in the sequence when processing each element, thereby capturing global dependencies. The cross-attention mechanism can promote the interaction between the image feature encoding and the task information encoding, so that the model can dynamically adjust its attention to image information based on task requirements. After this step, the intermediate task-related features and intermediate image feature encodings are obtained.

[0090] In a feasible embodiment, step S201 may include steps S2011 to S2012:

[0091] Step S2011, performing self-attention mechanism processing on the preset task information encoding through the bidirectional transformer block to obtain a new task information encoding;

[0092] The task information encoding serves as an instruction or identifier to guide the model to perform a specific task. It is processed by the self-attention mechanism in the bidirectional transformer block to promote the deep integration of internal task information and generate a new task information encoding that is richer and more contextually relevant. The new task information encoding not only retains the meaning of the original task information encoding, but also incorporates the contextual information of other encodings in the sequence.

[0093] Step S2012, cross-attention mechanism processing is performed on the new task information encoding, image feature encoding and preset image position encoding to obtain intermediate task-related features and intermediate image feature encoding.

[0094] In the bidirectional transformer block, the new task information encoding, image feature encoding, and image position encoding are processed by the cross-attention mechanism. At this stage, a dynamic interactive relationship is established between the new task information encoding and the image feature encoding. The cross-attention mechanism enables the model to selectively focus on specific areas or features in the image according to task requirements. At the same time, the introduction of image position encoding maintains the position information of the information in the sequence, which helps the model understand the relative relationship between elements. After this process, the intermediate task-related features and intermediate image feature encoding are obtained.

[0095] Step S202, input the intermediate task-related features and the intermediate image feature codes into the cross-attention sub-unit for cross-attention mechanism processing to obtain task-related features and updated image feature codes.

[0096] The intermediate task-related features and intermediate image feature encodings are taken as input and passed to the cross-attention sub-unit, which further uses the cross-attention mechanism to deepen the interaction between image feature encodings and task-related features. Through this processing, the model can more finely identify the parts of the face image that are most relevant to the face analysis task, and generate the final task-related features and update the image feature encoding based on this.

[0097] For example, in a feasible implementation, the structural diagram of the decoding attention unit is as follows: Figure 3 As shown in the figure, the decoding attention unit includes two bidirectional transformer blocks and a cross attention mechanism. The input of each bidirectional transformer block is image feature encoding, image position encoding, learnable task-related features and task information encoding information. The task information encoding information is hidden in the initial task-related features. The image feature encoding output by the bidirectional transformer block 1 is the intermediate image feature encoding, and the task-related features output by the bidirectional transformer block 1 are the intermediate task-related features. Then, the intermediate task-related features and the intermediate image feature encoding are input to the cross attention sub-unit for cross attention mechanism processing to obtain the output task-related features and the output image feature encoding for the subsequent decoding output unit.

[0098] The structural diagram of the bidirectional transformer block is as follows Figure 4As shown in FIG. 1 , the structures of bidirectional transformer block 0 and bidirectional transformer block 1 are basically the same. The difference between the two lies in the difference in the steps of processing the task-related features with the self-attention mechanism. Specifically, bidirectional transformer block 0 processes the task-related features separately with the self-attention mechanism to obtain new task-related features, while bidirectional transformer block 1 processes the task-related features with the self-attention mechanism based on the task-related features processed by bidirectional transformer block 0 and combined with the task-related features to obtain new task-related features. For bidirectional transformer block 0 or bidirectional transformer block 1, firstly, self-attention calculation is performed on the learnable task-related features to capture the correlation information between the tasks. Then, the cross-attention module of the task to the image and the cross-attention module of the image to the task complete the coupling of the task information and the image information, so that the model learns the corresponding relationship between the task and the image features. The results of the cross-attention are normalized after adding the residual. In this process, the image features and the task-related features are continuously updated with the forward reasoning, and the image position encoding and the task image position encoding save the position information and remain unchanged.

[0099] In a feasible embodiment, the subtasks include face key point prediction, face posture prediction, face age prediction, face gender prediction and face race prediction, and step S30 may include steps S301 to S302:

[0100] Step S301, decomposing the task-related features into sub-task-related features through a decoding output unit, the sub-task-related features including face key point features, face posture features, face age features, face gender features and face ethnicity features;

[0101] The decoding output unit decomposes the task-related features into sub-task-related features with clear directionality, which can be specifically decomposed into facial key point features, facial posture features, facial age features, facial gender features and facial race features. These sub-task-related features each carry key information closely related to the corresponding sub-task.

[0102] Step S302, after subjecting the facial key point features, facial posture features, facial age features, facial gender features, and facial ethnicity features to multi-layer MLP layer feature conversion processing, the facial key point prediction, facial posture prediction, facial age prediction, facial gender prediction, and facial ethnicity prediction are obtained. The corresponding facial analysis results;

[0103] In order to extract more accurate analysis results from the subtask related features, the multi-layer perceptron (MLP) layer is used to perform feature conversion processing on the facial key point features, facial posture features, and facial age, gender, and ethnicity features, and the corresponding facial analysis results of facial key point prediction, facial posture prediction, facial age, gender, and ethnicity prediction are obtained. These facial analysis results are not only highly accurate, but also highly practical and can be widely used in various facial analysis scenarios.

[0104] In a feasible embodiment, the task decoding module further includes an upsampling unit, the subtask further includes face semantic segmentation prediction, and step S30 may include steps S303 to S306:

[0105] Step S303, decomposing the task-related features into sub-task-related features through a decoding output unit, where the sub-task-related features include face semantic segmentation features;

[0106] The decoding output unit decomposes the task-related features into sub-task-related features with clear directionality, and the sub-task-related features include face semantic segmentation features.

[0107] Step S304, performing multi-layer MLP layer feature conversion processing on the face semantic segmentation features to obtain updated face semantic segmentation features;

[0108] In order to extract more accurate analysis results from subtask related features, the multi-layer perceptron (MLP) layer is applied to the face semantic segmentation features for feature conversion processing to obtain the face analysis results corresponding to the face semantic segmentation prediction.

[0109] Step S305, performing an upsampling operation on the updated image feature code by an upsampling unit to obtain an updated image feature code matrix;

[0110] The update image feature coding is upsampled by the upsampling unit, so as to obtain an update image feature coding matrix with a resolution consistent with the input size of the face image and richer details.

[0111] Step S306, multiplying the updated face semantic segmentation feature and the updated image feature encoding matrix to obtain the face analysis result corresponding to the face semantic segmentation prediction.

[0112] After obtaining the updated image feature encoding matrix and the face semantic segmentation features, the updated image feature encoding matrix and the face semantic segmentation features are multiplied to obtain the face analysis result corresponding to the face semantic segmentation prediction.

[0113] It should be noted that since the attention mechanism on which the transformer model relies does not have the ability to encode image positions, it is necessary to use image position encoding to solve the position information problem. In this embodiment, a random Gaussian image position encoding matrix is ​​used as the position information input. At the same time, for the multi-task face analysis problem, each task is represented as a learnable code in this embodiment, that is, initialized as a learnable feature vector, and they are processed together to improve the generalization ability and task synergy across tasks. Therefore, the input of the decoding module includes the image feature code obtained by multi-scale feature extraction, the image position code composed of position information, and the task information code composed of task information.

[0114] For example, the structural diagram of the task decoding module is as follows: Figure 5As shown, the task decoding module includes an input unit, a decoding attention unit and a decoding output unit, wherein the input unit receives the image position code generated by the image position coding unit, the task information code generated by the task information coding unit and the image feature code generated by the multi-scale feature coding module. The task information coding unit can obtain the task information code based on the corresponding codes of each sub-task of the determined face analysis, for example, the corresponding codes of each sub-task include the task code of face key point prediction, the task code of face posture prediction, the task code of face age prediction, the task code of face gender prediction, the task code of face race prediction and the task code of face semantic segmentation; the multi-scale feature coding module can generate the image feature code according to the input face image, the image position code is used to add the image position code to the feature to retain the spatial information, and the task information code is used to add a specific tag embedding to the task. Then, by decoding the attention unit, the task-specific information encoding (i.e., task-related features) and the updated image feature information (i.e., updated image feature encoding) are obtained. Then, the task-specific information encoding is decomposed into sub-task-related features, including face key point features, face posture features, face age features, face gender features, face race features, and face semantic segmentation features. Each feature is input into the corresponding prediction task head to output the face analysis result. Among them, the prediction task head may include a face key point prediction task head, a face posture prediction task head, a face age prediction task head, a face gender prediction task head, a face race prediction task head, and a face semantic segmentation task head. The upsampling unit in the task decoding module performs an upsampling operation on the update image feature coding to obtain an updated image feature coding matrix, and multiplies the updated image feature coding matrix and the face semantic segmentation features updated by the multi-layer MLP (output by the face semantic segmentation task head) to obtain the face analysis result corresponding to the face semantic segmentation task. In addition, the face key point prediction task head, face posture prediction task head, face age prediction task head, face gender prediction task head, and face race prediction task head input the relevant features of each subtask, and after multi-layer MLP layer feature conversion, the face analysis result corresponding to the subtask is output.

[0115] For example, the output information of the prediction task head corresponding to each subtask is as follows:

[0116] Face key point prediction task head: Input the feature information related to the face key point prediction task, and after multi-layer MLP layer feature conversion, output 136 key points, representing the coordinates of 68 face key points.

[0117] Face pose prediction task head: inputs the feature information related to the face pose prediction task, and after multi-layer MLP layer feature conversion, outputs the 6D rotation matrix of pose estimation.

[0118] Face age prediction task head: inputs the feature information related to the face age estimation task, and after multi-layer MLP layer feature conversion, outputs the predicted probability of belonging to each age group. In this embodiment, according to the classification method of the Fairface dataset, human age is divided into 9 categories, namely 0-2, 3-9, 10-19, 20-29, 30-29, 40-49, 50-59, 60-69 and 70 and above.

[0119] Face gender prediction task head: Input the feature information related to the face gender estimation task, and after multi-layer MLP layer feature conversion, output the binary classification probability of male and female.

[0120] Face race prediction task head: inputs the feature information related to the face race estimation task, and after multi-layer MLP layer feature conversion, outputs the predicted probability of belonging to each race. In this embodiment, according to the classification method of the Fairface dataset (an open source dataset for facial attribute recognition detection), face race is divided into 7 categories.

[0121] Face semantic segmentation task head: input the feature information related to the face semantic segmentation task and the image feature encoding information output by the attention module, output the face semantic information after the multi-layer MLP layer feature conversion, multiply it with the upsampled image feature encoding information matrix, and output the final face semantic segmentation map, the size of which is consistent with the input image, and the upsampling layer is implemented using deconvolution. In this embodiment, the segmentation categories are background, skin, right eyebrow, left eyebrow, right eye, left eye, nose, upper lip, mouth, lower lip, and hair. The segmentation categories can be modified as needed in the application, and this embodiment does not limit them.

[0122] At this point, we input a face image and can obtain the multi-task face analysis result of the face image.

[0123] In a feasible embodiment, before step S10, steps A10 to A40 may be further included:

[0124] Step A10, obtaining training samples, where the training samples include a plurality of labeled training face images;

[0125] Step A20, inputting the training face image into the initial face analysis model to obtain the training face analysis result;

[0126] Acquire training samples, which include multiple labeled training face images, input the training face images into the initial face analysis model, and obtain training face analysis results. Specifically, extract features from these training samples through the initial face analysis model to obtain face feature vectors, which include face geometric structure information and multi-scale visual features such as texture and color. Then, input the extracted face feature vectors into the initial task decoding module in the initial face analysis model, and the initial task decoding module decodes the input face feature vectors to obtain training face analysis results. The training face analysis results include a variety of information such as the location of facial key points, facial posture, age, gender, race, and facial semantic segmentation, so as to provide data support for subsequent training and evaluation.

[0127] Step A30, calculating a joint loss function according to the training face analysis results and the true similarity values ​​of the training samples, wherein the joint loss function is obtained by weighted summing up the face key point prediction task loss function, the face posture prediction task loss function, the face age prediction task loss function, the face gender prediction task loss function, the face race prediction task loss function and the face semantic segmentation task loss function;

[0128] In order to evaluate the performance of the initial face analysis model and optimize it, a joint loss function is calculated based on the training face analysis results and the true similarity values ​​of the training samples. The joint loss function is obtained by weighted summation of multiple subtask loss functions, including the face key point prediction task loss function, the face posture prediction task loss function, the face age, gender, race prediction task loss function, and the face semantic segmentation task loss function. The weighted value of each sub-function loss function can be set according to the actual application scenario, and this is not specifically limited in this embodiment. Through the calculation of this joint loss function, the performance of the initial face analysis model on each subtask can be comprehensively and accurately evaluated, and clear guidance can be provided for its subsequent optimization.

[0129] Step A40, updating the parameters of the initial face analysis model according to the joint loss function to obtain a trained face analysis model.

[0130] The parameters of the initial face analysis model are updated according to the calculated joint loss function. Through continuous iteration and optimization, a trained face analysis model is obtained. This face analysis model not only performs well in the prediction of each subtask, but also has stronger generalization ability and robustness, and can more accurately cope with various complex face analysis tasks.

[0131] In a feasible embodiment, before step A30, steps A50 to A80 may also be included:

[0132] Step A50, constructing a face key point prediction task loss function based on the Wingloss loss function;

[0133] Step A60, constructing a face pose prediction task loss function based on the rotation matrix geodesic distance loss function;

[0134] Step A70, constructing a face age prediction task loss function, a face gender prediction task loss function, and a face race prediction task loss function based on the cross entropy loss function;

[0135] Step A80, constructing a face semantic segmentation task loss function based on the second cross entropy loss function.

[0136] Specifically, the weighted loss functions of multiple tasks are used to obtain the joint loss function. The specific expression of the joint loss function is as follows:

[0137] L=λ land loss land +λ pose loss pose +λ age loss age +λ hende loss gender +

[0138] λ race loss race +λ seg loss seg ;

[0139] Where λ represents the weight of each subtask loss, and the loss corresponding to each subtask is calculated as follows:

[0140] loss land It represents the loss of the facial key point prediction task. In this embodiment, 68 key points are used to represent the facial feature point positioning of the face, that is, N=68 in the following formula, and Wingloss (a loss function algorithm) is used to calculate the key point prediction loss.

[0141]

[0142] Where (x i ,y i ) is the real face key point label, is the network prediction output, ω,∈, is the hyperparameter of the function, which is a constant during training. In this embodiment, ω=10.0, ∈=2.0, C is a constant calculated from ω and ∈, C=ωln(1+ω / ∈).

[0143] loss poseRepresents the face pose prediction loss. The output of the network in this task is a 6D rotation matrix of pose estimation with a size of batch*6. Before calculating the loss, the 6D pose needs to be converted to a batch*3*3 rotation matrix. The conversion formula is as follows:

[0144] a1 = pose_out[:,0:3];

[0145] a2 = pose_out[:,3:6];

[0146] Where pose_out is the 6D rotation matrix of the network output, a1 represents the first 3 columns of pose_out, and a2 represents the last 3 columns of pose_out;

[0147]

[0148] Among them, |a1| is the modulus of a1, r1 is the normalized result of a1, and the size is batch*3;

[0149]

[0150] Among them, r2 is the cross product of r1 and a2, and then normalized, and the size is batch*3;

[0151] r3=r1×r2;

[0152] R p =[r1r2r3];

[0153] Among them, r3 is the result of the cross product of r1 and r2, and its size is batch*3. The rotation matrix R is obtained by concatenating r1, r2, and r3. p , the size is batch*3*3.

[0154] Then use the rotation matrix to calculate the true rotation matrix R gt With the predicted rotation matrix R p The geodesic distance is used as the face pose prediction loss:

[0155]

[0156] Among them, tr(R p R gt T ) is used to find the trace of the multiplication of the two matrices, and then the arc cosine value is calculated. The output is in radians.

[0157] The face age, gender, and race estimation are all classification tasks. In this embodiment, the cross entropy loss is used to calculate the gap between the predicted value and the true label. For a single sample, the loss function is specifically expressed as:

[0158]

[0159] Among them, y i is a one-hot encoding of the true label, is the predicted value of each category, and C is the number of categories for each task.

[0160] loss seg Indicates the face semantic segmentation loss. The output of the network in this task is a face semantic segmentation map with the same size as the input image. The number of channels is 11, which respectively represents the probability of each pixel belonging to each category. In this example, OhemCELoss (Onlinehardexampleminingcross-entropyloss, a cross entropy loss function) loss is used. In essence, the cross entropy loss of each pixel is calculated, and the minimum number of pixels n is set. min And the loss threshold thresh, if the first n min If the smallest loss among the n losses is still greater than the set threshold, then all elements greater than the threshold are taken to calculate the loss. Otherwise, the first n losses are calculated. min The loss value of pixels. Here n min and thresh are hyperparameters in training. In this paper, n min =batch*256*256 / 16, thresh=0.7, other values ​​may also be used without specific limitation.

[0161] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the face analysis method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0162] The embodiment of the present application also provides a face analysis system, which performs face analysis based on a face analysis model. The face analysis model includes a multi-scale feature encoding module and a task decoding module. The task decoding module includes a decoding attention unit and a decoding output unit. Please refer to Figure 6 , the face analysis system includes:

[0163] The feature acquisition module 10 is used to input the face image into the multi-scale feature encoding module to obtain the image feature encoding;

[0164] A feature updating module 20, used to input the image feature code, the preset task information code and the preset image position code into the decoding attention unit to obtain the task-related features and update the image feature code;

[0165] The face analysis module 30 is used to decompose the task-related features into sub-task-related features through a decoding output unit, and obtain the face analysis results corresponding to each sub-task according to the sub-task-related features and the updated image feature encoding.

[0166] Optionally, the multi-scale feature encoding module includes a multi-scale face feature acquisition unit and a feature fusion unit, and the feature acquisition module 10 is further used to:

[0167] Inputting the face image into a multi-scale face feature acquisition unit to obtain face features of multiple different scales;

[0168] Each facial feature is input into the feature fusion unit for feature fusion processing to obtain image feature encoding with uniform scale.

[0169] Optionally, the task decoding module further includes an image position encoding unit and a task information encoding unit, and the feature updating module 20 is further used to:

[0170] Generate an image position code by an image position coding unit;

[0171] Through the task information encoding unit, each subtask is initialized as a learnable feature vector, and the feature vectors are concatenated to obtain the task information encoding.

[0172] Optionally, the decoding attention unit includes a bidirectional transformer block and a cross attention sub-unit, and the feature updating module 20 is further used to:

[0173] The image feature encoding, the preset task information encoding and the preset image position encoding are input into the bidirectional transformer block for self-attention mechanism and cross-attention mechanism processing to obtain the intermediate task-related features and the intermediate image feature encoding;

[0174] The intermediate task-related features and intermediate image feature codes are input into the cross-attention subunit for cross-attention mechanism processing to obtain task-related features and updated image feature codes.

[0175] Optionally, the feature updating module 20 is further configured to:

[0176] The preset task information encoding is processed by the self-attention mechanism through the bidirectional transformer block to obtain a new task information encoding;

[0177] The new task information encoding, image feature encoding and preset image position encoding are processed by the cross-attention mechanism to obtain intermediate task-related features and intermediate image feature encoding.

[0178] Optionally, the subtasks include face key point prediction, face posture prediction, face age prediction, face gender prediction and face ethnicity prediction. The face analysis module 30 is further used for:

[0179] The task-related features are decomposed into sub-task-related features through the decoding output unit. The sub-task-related features include facial key point features, facial posture features, facial age features, facial gender features and facial ethnicity features.

[0180] After multi-layer MLP layer feature conversion processing of facial key point features, facial posture features, facial age features, facial gender features and facial race features, we obtain the corresponding face analysis results of facial key point prediction, facial posture prediction, facial age prediction, facial gender prediction and facial race prediction.

[0181] Optionally, the task decoding module further includes an upsampling unit, and the subtask further includes face semantic segmentation prediction, and the face analysis module 30 is further used for:

[0182] Decomposing the task-related features into sub-task-related features through the decoding output unit, the sub-task-related features include face semantic segmentation features;

[0183] The face semantic segmentation feature is subjected to multi-layer MLP layer feature conversion processing to obtain an updated face semantic segmentation feature;

[0184] An upsampling operation is performed on the updated image feature coding by an upsampling unit to obtain an updated image feature coding matrix;

[0185] The updated face semantic segmentation features and the updated image feature encoding matrix are multiplied to obtain the face analysis result corresponding to the face semantic segmentation prediction.

[0186] Optionally, the face analysis system further includes a model training module (not shown), which is used to:

[0187] Obtaining training samples, where the training samples include a plurality of labeled training face images;

[0188] Input the training face image into the initial face analysis model to obtain the training face analysis result;

[0189] The joint loss function is calculated based on the training face analysis results and the true similarity values ​​of the training samples, wherein the joint loss function is obtained by weighted summing the face key point prediction task loss function, the face pose prediction task loss function, the face age prediction task loss function, the face gender prediction task loss function, the face race prediction task loss function and the face semantic segmentation task loss function;

[0190] The parameters of the initial face analysis model are updated according to the joint loss function to obtain a trained face analysis model.

[0191] Optionally, the model training module is also used to:

[0192] Construct the loss function of face key point prediction task based on Wingloss loss function;

[0193] Construct the loss function of face pose prediction task based on the loss function of rotation matrix geodesic distance;

[0194] Based on the cross entropy loss function, we construct the face age prediction task loss function, face gender prediction task loss function, and face race prediction task loss function respectively;

[0195] The loss function of the face semantic segmentation task is constructed based on the second cross entropy loss function.

[0196] The face analysis system provided in the embodiment of the present application adopts the face analysis method in the above embodiment, and can improve the overall performance of the face analysis system by improving the analysis accuracy and system integration efficiency of the face analysis system. Compared with the prior art, the beneficial effects of the face analysis system provided in the embodiment of the present application are the same as the beneficial effects of the face analysis method provided in the above embodiment, and other technical features of the face analysis system are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0197] An embodiment of the present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the face analysis method in the above-mentioned embodiment 1.

[0198] Reference below Figure 7 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic devices in the embodiments of the present application may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The verification value calculation device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0199] like Figure 7 As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0200] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0201] The electronic device provided in the embodiment of the present application adopts the face analysis method in the above embodiment, which can improve the overall performance of the face analysis system by improving the analysis accuracy and system integration efficiency of the face analysis system. Compared with the prior art, the beneficial effects of the electronic device provided in the embodiment of the present application are the same as the beneficial effects of the face analysis method provided in the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0202] It should be understood that the various parts disclosed in the embodiments of the present application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0203] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0204] An embodiment of the present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the face analysis method in the above-mentioned embodiment.

[0205] The computer-readable storage medium provided in the embodiment of the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0206] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0207] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the electronic device: inputs the face image into the multi-scale feature coding module to obtain the image feature code; inputs the image feature code, the preset task information code and the preset image position code into the decoding attention unit to obtain the task-related features and the updated image feature code; decomposes the task-related features into each sub-task-related features through the decoding output unit, and obtains the face analysis results corresponding to each sub-task according to each sub-task-related features and the updated image feature code.

[0208] The computer program code for performing the operation of the embodiment of the present application can be written in one or more programming languages ​​or a combination thereof, and the above-mentioned programming languages ​​include object-oriented programming languages, such as Java, Smalltalk, C++, and also include conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN: Local Area Network) or a wide area network (WAN: Wide Area Network), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0209] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0210] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0211] The readable storage medium provided in the embodiment of the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned face analysis method, and can improve the overall performance of the face analysis system by improving the analysis accuracy and system integration efficiency of the face analysis system. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the embodiment of the present application are the same as the beneficial effects of the face analysis method provided in the above-mentioned embodiment, and will not be repeated here.

[0212] The above are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A face analysis method, characterized in that: The face analysis method performs face analysis based on a face analysis model, wherein the face analysis model includes a multi-scale feature encoding module and a task decoding module, wherein the task decoding module includes a decoding attention unit and a decoding output unit, and the face analysis method includes: Inputting the face image into the multi-scale feature encoding module to obtain image feature encoding; Inputting the image feature code, the preset task information code and the preset image position code into the decoding attention unit to obtain task-related features and update the image feature code; The task-related features are decomposed into sub-task-related features by the decoding output unit, and the face analysis results corresponding to each sub-task are obtained according to the sub-task-related features and the updated image feature encoding.

2. The face analysis method according to claim 1, characterized in that: The multi-scale feature encoding module includes a multi-scale face feature acquisition unit and a feature fusion unit. The step of inputting the face image into the multi-scale feature encoding module to obtain the image feature encoding includes: Inputting the facial image into the multi-scale facial feature acquisition unit to obtain facial features of multiple different scales; Each of the facial features is input into the feature fusion unit for feature fusion processing to obtain image feature encoding with unified scale.

3. The face analysis method according to claim 1, characterized in that: The task decoding module further includes an image position encoding unit and a task information encoding unit. Before the steps of inputting the image feature code, the preset task information code and the preset image position code into the decoding attention unit, obtaining task-related features and updating the image feature code, the module further includes: Generate an image position code by the image position coding unit; Through the task information encoding unit, each subtask is initialized as a learnable feature vector, and each feature vector is concatenated to obtain the task information encoding.

4. The face analysis method according to claim 1, characterized in that: The decoding attention unit includes a bidirectional transformer block and a cross attention sub-unit, and the step of inputting the image feature code, the preset task information code and the preset image position code into the decoding attention unit to obtain task-related features and update the image feature code includes: Inputting the image feature code, the preset task information code and the preset image position code into the bidirectional transformer block for self-attention mechanism and cross-attention mechanism processing to obtain intermediate task-related features and intermediate image feature codes; The intermediate task-related features and the intermediate image feature codes are input into the cross-attention subunit for cross-attention mechanism processing to obtain task-related features and updated image feature codes.

5. The face analysis method according to claim 4, characterized in that: The step of inputting the image feature code, the preset task information code and the preset image position code into the bidirectional transformer block for self-attention mechanism and cross-attention mechanism processing to obtain intermediate task-related features and intermediate image feature codes comprises: The preset task information encoding is processed by the self-attention mechanism through the bidirectional transformer block to obtain a new task information encoding; The new task information encoding, the image feature encoding and the preset image position encoding are processed by a cross-attention mechanism to obtain intermediate task-related features and intermediate image feature encodings.

6. The face analysis method according to claim 1, characterized in that: The subtasks include face key point prediction, face posture prediction, face age prediction, face gender prediction and face race prediction. The step of decomposing the task-related features into subtask-related features through the decoding output unit, and obtaining face analysis results corresponding to each subtask according to the subtask-related features and the updated image feature encoding includes: Decomposing the task-related features into sub-task-related features through the decoding output unit, wherein the sub-task-related features include facial key point features, facial posture features, facial age features, facial gender features, and facial ethnicity features; After the facial key point features, the facial posture features, the facial age features, the facial gender features and the facial race features are respectively processed by multi-layer MLP layer feature conversion, the facial analysis results corresponding to the facial key point prediction, the facial posture prediction, the facial age prediction, the facial gender prediction and the facial race prediction are obtained.

7. The face analysis method according to claim 6, characterized in that: The task decoding module further includes an upsampling unit, the subtask further includes face semantic segmentation prediction, and the step of decomposing the task-related features into subtask-related features through the decoding output unit, and obtaining face analysis results corresponding to each subtask according to the subtask-related features and the updated image feature encoding includes: Decomposing the task-related features into sub-task-related features through the decoding output unit, wherein the sub-task-related features include face semantic segmentation features; The face semantic segmentation feature is subjected to multi-layer MLP layer feature conversion processing to obtain an updated face semantic segmentation feature; Performing an upsampling operation on the updated image feature code by the upsampling unit to obtain an updated image feature code matrix; The updated face semantic segmentation feature is multiplied by the updated image feature encoding matrix to obtain a face analysis result corresponding to the face semantic segmentation prediction.

8. The face analysis method according to claim 1, characterized in that: Before the step of inputting the face image into the multi-scale feature encoding module to obtain the image feature encoding, the method further includes: Acquire a training sample, wherein the training sample includes a plurality of labeled training face images; Inputting the training face image into an initial face analysis model to obtain a training face analysis result; Calculating a joint loss function according to the training face analysis result and the true similarity value of the training sample, wherein the joint loss function is obtained by weighted summation of a face key point prediction task loss function, a face posture prediction task loss function, a face age prediction task loss function, a face gender prediction task loss function, a face race prediction task loss function, and a face semantic segmentation task loss function; The parameters of the initial face analysis model are updated according to the joint loss function to obtain a trained face analysis model.

9. The face analysis method according to claim 8, characterized in that: Before the step of calculating the joint loss function according to the training face analysis result and the true similarity value of the training sample, the method further includes: Constructing the face key point prediction task loss function based on the Wingloss loss function; Constructing the face pose prediction task loss function based on the rotation matrix geodesic distance loss function; Based on the cross entropy loss function, the face age prediction task loss function, the face gender prediction task loss function and the face race prediction task loss function are respectively constructed; The face semantic segmentation task loss function is constructed based on the second cross entropy loss function.

10. An electronic device, characterized in that: The electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the face analysis method according to any one of claims 1 to 9.

11. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the face analysis method according to any one of claims 1 to 9 are implemented.

Images