A modular facial attribute structure analysis method, apparatus, device and medium

Abstract

The application relates to a modular face attribute structure analysis method and device, equipment and medium, and relates to the technical field of image recognition processing. The method comprises the following steps: performing face detection alignment on a face image to be analyzed according to a preset detection task to obtain an aligned face image, selecting a target task module from an attribute structure analysis model according to the detection task, performing attribute analysis and extraction on the aligned face image through the target task module to obtain a single attribute analysis result set, extracting features from the aligned face image through each feature extraction module in a backbone model to obtain an attribute analysis result set and a face feature set, performing gradient propagation according to the attribute analysis result set and the face feature set to obtain a face attribute analysis result corresponding to the face image to be analyzed, and the application can flexibly complete diversified face attribute detection tasks by combining different task modules, thereby realizing efficient face attribute analysis and improving model recognition accuracy.

Description

Technical Field

[0001] This application relates to the field of image recognition and processing technology, and in particular to a modular method, apparatus, device and medium for analyzing facial attributes and structures. Background Technology

[0002] Face attribute detection is a complex computer vision task with wide applications in many fields such as security monitoring. It typically requires algorithms to accurately identify and classify multiple attributes in face images. Its complexity lies in the fact that face attribute detection not only requires the model to accurately identify multiple features of an individual, but also needs to maintain stable performance under different environments and conditions.

[0003] Existing facial attribute analysis solutions mainly involve building multi-attribute classification models to predict all attributes simultaneously. However, there are multiple facial attributes, and facial attribute detection also involves multiple classification tasks. These tasks typically require models with high complexity and a large number of parameters to ensure that they can capture subtle differences in the face. Multi-attribute classification models need to learn multiple complex tasks simultaneously within a limited parameter space, which can easily lead to overfitting or underfitting, resulting in low model recognition accuracy. Summary of the Invention

[0004] This application provides a modular face attribute structure analysis method, apparatus, device, and medium. By combining different task modules, it can flexibly complete diverse face attribute detection tasks. The required detection items can be selected and combined in a modular manner, and then the task modules required to process the detection items can be selected, thereby achieving efficient face attribute analysis, reducing computational resource consumption, and solving the problem of low recognition and detection accuracy in existing face attribute analysis methods that use multi-attribute classification models for detection.

[0005] Firstly, this application provides a modular method for analyzing facial attribute structures, including:

[0006] Obtain the face image to be analyzed;

[0007] According to the preset detection task, the face image to be analyzed is aligned to obtain an aligned face image.

[0008] According to the detection task, at least one target task module is selected from the preset attribute structure analysis model, and the aligned face image is input into the target task module and the backbone model in the attribute structure analysis model.

[0009] The aligned face image is analyzed and extracted by each of the target task modules to obtain a set of single attribute analysis results corresponding to the detection task, and the aligned face image is extracted by each feature extraction module in the backbone model to obtain a face feature set.

[0010] Gradient propagation is performed based on the single attribute analysis result set and the face feature set to obtain the face attribute analysis result corresponding to the face image to be analyzed.

[0011] Optionally, the step of performing face detection and alignment on the face image to be analyzed according to a preset detection task to obtain an aligned face image includes:

[0012] Obtain the preset detection task and the preset background feature key points corresponding to the detection task;

[0013] Face detection is performed on the face image to be analyzed, and key alignment positions are selected from the face detection results according to the detection task.

[0014] By using the background feature key points and combining the key alignment positions, the face image to be analyzed is aligned to obtain an aligned face image.

[0015] The background feature key points are key points containing objects around the face, and the aligned face image includes face features and face background features.

[0016] Optionally, the step of performing attribute analysis and extraction on the aligned face image through each of the target task modules to obtain a single attribute analysis result set corresponding to the detection task, and performing feature extraction on the aligned face image through each feature extraction module in the backbone model to obtain a face feature set, includes:

[0017] Determine the first arrangement structure of each module in the attribute structure analysis model. The first arrangement structure is the order of the target task modules. The target task modules correspond one-to-one with the feature extraction modules.

[0018] According to the first arrangement structure, the aligned face image is sequentially analyzed and extracted by the target task module to obtain a single attribute analysis result set, and then the aligned face image is sequentially identified and extracted by the feature extraction module to obtain a face feature set.

[0019] Optionally, the step of performing gradient propagation based on the single-attribute analysis result set and the face feature set to obtain the face attribute analysis result corresponding to the face image to be analyzed includes:

[0020] According to the arrangement structure, the face attribute analysis results are obtained by forward propagation using the single attribute analysis results in the single attribute analysis result set and the face features in the face feature set.

[0021] Optionally, before selecting at least one corresponding target task module from the preset attribute structure analysis model, the method further includes:

[0022] Acquire a facial attribute dataset and construct an initial analysis model. The initial analysis model includes a single-task model and a backbone model. The single-task model includes task modules that correspond one-to-one with each feature extraction module in the backbone model. The feature extraction modules and the task modules call each other.

[0023] Based on the preset background feature key points, face detection and face alignment are performed on the sample face images in the face attribute dataset to obtain aligned face image samples.

[0024] Select a task module corresponding to a preset attribute detection item from the single-task model as a task module to be trained, and input the aligned face image sample into the task module to be trained and the backbone model;

[0025] The alignment face image samples are used to train the task module to be trained by attribute analysis, and the single prediction result output by the task module to be trained is obtained. The feature prediction result output by the backbone model for the alignment face image samples is also obtained.

[0026] Gradient propagation is performed based on the individual prediction results and the feature prediction results to obtain the model prediction results;

[0027] The loss is calculated using the real labels corresponding to the sample face images and the model prediction results. The parameters of the task module to be trained are optimized based on the loss calculation results until the single-task model is trained, resulting in a trained attribute structure analysis model.

[0028] Optionally, before inputting the aligned face image samples into the training task module and the backbone model, the method further includes:

[0029] Obtain the face recognition dataset;

[0030] The feature extraction modules in the backbone model are trained using the face recognition dataset, and the parameters of the feature extraction results output by the feature extraction modules are optimized using a preset loss function until the loss converges, thus obtaining the trained backbone model.

[0031] The parameters of the backbone model are fixed after pre-training, and the backbone model does not participate in the attribute analysis training of the task module to be trained.

[0032] Optionally, the task module to be trained includes a first matrix and a second matrix, and the step of using the aligned face image samples to perform attribute analysis training on the task module to be trained includes:

[0033] The model update bias is obtained by multiplying the first matrix and the second matrix.

[0034] Based on the model update bias, the aligned face image samples, and the initial parameters of the model, attribute analysis training is performed to obtain the single prediction result output by the task module to be trained.

[0035] Optionally, the step of performing attribute analysis training based on the model update bias, the aligned face image samples, and the initial model parameters includes:

[0036] according to Determine the individual prediction results;

[0037] in, Here, f represents the model's predicted value, and f is the model function. For the input aligned face image sample, For model parameters, These are the parameters of the original, untrained model. The parameters were varied to train the model and fit them using two low-rank matrices. and Multiplication means, These are the parameters for the forward propagation of the feature extraction module in the backbone model. These are the parameters for the forward propagation of the task module to be trained.

[0038] Optionally, the step of performing gradient propagation based on the individual prediction result and the feature prediction result to obtain the model prediction result includes:

[0039] according to Perform gradient propagation;

[0040] The backbone model includes an input feature extraction module and at least four residual feature extraction modules, while the single-task model includes at least five training task modules corresponding to the feature extraction modules of the backbone model. , , , as well as All of these are forward propagations of the feature extraction modules of the backbone model. , , , as well as All of these are forward propagations of each training task module in a single-task model. This represents the model's prediction results.

[0041] Secondly, this application provides a modular face attribute structure analysis device, comprising:

[0042] The image acquisition module is used to acquire the face image to be analyzed;

[0043] The face detection and alignment module is used to perform face detection and alignment on the face image to be analyzed according to a preset detection task, so as to obtain an aligned face image.

[0044] The input module is used to select at least one target task module from the preset attribute structure analysis model according to the detection task, and input the aligned face image into the target task module and the backbone model in the attribute structure analysis model;

[0045] The feature extraction module is used to perform attribute analysis and extraction on the aligned face image through each of the target task modules to obtain a set of single attribute analysis results corresponding to the detection task, and to perform feature extraction on the aligned face image through each feature extraction module in the backbone model to obtain a face feature set;

[0046] The gradient propagation module is used to perform gradient propagation based on the single attribute analysis result set and the face feature set to obtain the face attribute analysis result corresponding to the face image to be analyzed.

[0047] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0048] Memory, used to store computer programs;

[0049] When a processor executes a program stored in memory, it implements the steps of the modular face attribute structure analysis method as described in any embodiment of the first aspect.

[0050] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the modular face attribute structure analysis method as described in any embodiment of the first aspect.

[0051] In summary, this embodiment of the application acquires a face image to be analyzed, performs face detection alignment on the face image to be analyzed according to a preset detection task to obtain an aligned face image, and then selects at least one corresponding target task module from a preset attribute structure analysis model according to the detection task. The aligned face image is then input into the target task module and the backbone model in the attribute structure analysis model. Subsequently, attribute analysis and extraction are performed on the aligned face image through each target task module to obtain a single attribute analysis result set corresponding to the detection task, and feature extraction is performed on the aligned face image through each feature extraction module in the backbone model to obtain a face feature set. Gradient propagation is then performed based on the single attribute analysis result set and the face feature set to obtain the face attribute analysis result corresponding to the face image to be analyzed. As can be seen, the embodiments of this application set up multiple task modules and feature extraction modules corresponding to detection tasks in the attribute analysis model. By combining with different task modules, it can flexibly complete diverse face attribute detection tasks. It can select and combine the required detection items in a modular manner according to different business needs, thereby achieving efficient and customized face attribute analysis, improving the model recognition accuracy, and solving the problem of low recognition and detection accuracy in existing face attribute analysis methods when using multi-attribute classification models for detection. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 A flowchart illustrating a modular face attribute structure analysis method provided in this application embodiment;

[0055] Figure 2 This is an optional example of a face alignment image comparison diagram provided in this application;

[0056] Figure 3 This is an optional example of a flowchart for facial attribute structure analysis provided in this application;

[0057] Figure 4 This is a schematic diagram illustrating the steps of training a modular face attribute structure analysis method model, provided in an optional embodiment of this application.

[0058] Figure 5This is a structural composition diagram of a multi-modular attribute structure analysis model provided in an optional embodiment of this application;

[0059] Figure 6 A structural block diagram of a modular face attribute structure analysis device provided in this application embodiment;

[0060] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] To facilitate understanding of the embodiments of this application, further explanations and descriptions will be provided below in conjunction with the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of this application.

[0063] Figure 1 This is a flowchart illustrating a modular face attribute structure analysis method provided in an embodiment of this application. Figure 1 As shown, the modular face attribute structure analysis method provided in this application embodiment may specifically include the following steps:

[0064] Step 110: Obtain the face image to be analyzed.

[0065] Step 120: Perform face detection and alignment on the face image to be analyzed according to the preset detection task to obtain an aligned face image.

[0066] Steps 110-120 are described uniformly as follows:

[0067] Specifically, the face image to be analyzed refers to an image containing a face. Preferably, the face image to be analyzed can contain a complete face and part of the background to retain more facial details and feature details of objects around the face in the background, thereby improving the accuracy of face alignment. The detection task refers to the detection items. In face attribute analysis, the detection items may include, but are not limited to: gender, age, wearing a mask, wearing a hat or headscarf, wearing glasses or sunglasses, whether makeup is worn, and expression detection. The detection items to be analyzed and detected can be freely selected according to actual needs to form the detection task of the face image to be analyzed. This allows for the free selection of items to be detected according to different business needs, realizing customized face attribute analysis. This embodiment does not limit the specific combination and number of detection items.

[0068] In related technologies, the existing process of facial attribute analysis can be roughly understood as: face detection - attribute classification / analysis, that is, directly inputting the face image into the model for attribute classification. This attribute classification method is usually affected by the face's deviation, tilt, etc. due to shooting reasons, resulting in low attribute classification accuracy.

[0069] In this embodiment, the face attribute analysis process may include, but is not limited to: face detection - key point detection - face alignment - attribute classification / analysis. That is, before performing face attribute classification, this embodiment first performs face alignment on the face image to be analyzed according to the detection items, so that the face image can be aligned with the template, thereby making the model unaffected by pose during training or application.

[0070] In one optional embodiment, this application embodiment performs face detection alignment on the face image to be analyzed according to a preset detection task to obtain an aligned face image. Specifically, this may include: obtaining a preset detection task and preset background feature key points corresponding to the detection task; performing face detection on the face image to be analyzed, and selecting a key alignment position from the face detection results according to the detection task; aligning the face image to be analyzed using the background feature key points and the key alignment position to obtain an aligned face image; wherein, the background feature key points are key points containing objects around the face, and the aligned face image includes face features and face background features.

[0071] In practice, existing face alignment solutions typically utilize facial landmarks or standard template faces to form five key points commonly used in the industry. such as key points =[[38.2946,51.6963],[73.5318,51.5014],[56.0252,71.7366],[41.5493,92.3655],[70.7299,92.2041]]. However, the actual face images obtained usually contain background information in addition to the face. Using the industry-standard key point templates, which are mainly adapted for feature extraction in face recognition and comparison, focuses more on the features of the face and fails to take into account the diversity of face images and backgrounds, resulting in the phenomenon of key features being cropped.

[0072] To address the problems existing in current face alignment schemes, this embodiment fully considers the background features contained in the face image and pre-constructs key points for detection items. It fully considers detection item data such as hats, headscarves, and facial expressions, and adjusts the key points to form a background feature key point set corresponding to the detection items. For example, the background feature key point set can be E=[[43.2946, 56.6963], [68.5318, 56.5014], [56.0252, 71.7366], [46.5493, 87.3655], [65.7299, 87.2041]], referring to... Figure 2 , Figure 2 Image 'a' is the face image obtained by aligning existing keypoints. Figure 2 Image b is the face image obtained after aligning the key points using this method. Because background factors are fully considered, the face alignment is more accurate, which can effectively improve the accuracy of attribute analysis and detection in subsequent attribute analysis and detection.

[0073] Reference Figure 3 The flowchart of face attribute analysis shown in this embodiment first selects the key points for face alignment according to the detection task, namely the background feature key points. Then, according to the detection task, the alignment position corresponding to the detection task is selected in the face detection results to obtain the key alignment position. Then, the background feature key points are combined with the face key points (including the eyes, nose tip, and left and right corners of the mouth, a total of 5 key points) to perform affine transformation on the face image to be analyzed to obtain the aligned face image.

[0074] Step 130: According to the detection task, select at least one corresponding target task module from the preset attribute structure analysis model, and input the aligned face image into the target task module and the backbone model in the attribute structure analysis model.

[0075] In this embodiment, the attribute structure analysis model may include a task model and a backbone model. The task model may include at least one task module. In this embodiment, each task module can be understood as a single task module, mainly implementing a single detection task in face attribute analysis. The backbone model may include feature extraction modules corresponding one-to-one with the task modules, used to extract face features. This embodiment does not impose any restrictions on this. Preferably, in this embodiment, the task model may be a LoRA (Lower-order adaptation of large language models) single-task model; the backbone model may be a ResNet model built using the ResNet framework. Preferably, in this embodiment, the ResNet50 model is selected as the backbone model for feature extraction.

[0076] In a specific implementation, this embodiment can input the aligned face image into the attribute structure analysis model. The attribute structure analysis model selects the task module corresponding to the detection item from each task model according to the detection item, as the target task module, and selects the corresponding feature extraction module for feature extraction for the target task module.

[0077] In related technologies, the complexity of facial attribute detection lies in the fact that it requires models not only to accurately identify multiple individual features but also to maintain stable performance under different environments and conditions. Facial attribute detection involves multiple classification tasks, including but not limited to age estimation, gender recognition, eye status (open or closed eyes), lip status (open or closed mouth), ear visibility, wearing glasses or sunglasses, wearing hats or headscarves, and makeup status. These tasks typically require models with high complexity and a large number of parameters to ensure the ability to capture subtle differences in the face. However, this complexity often comes with higher computational resource consumption, which is particularly problematic on resource-constrained devices. Existing facial attribute analysis methods, besides building multi-attribute classification models to predict all attributes simultaneously, also include some schemes that use multiple single-attribute classification models for detection. However, facial attributes are complex and diverse, and building a corresponding model for each attribute classification usually requires multiple models with high complexity and a large number of parameters, which significantly increases computational costs, including computational resource consumption during model training and inference. It is evident that existing facial attribute analysis methods cannot balance accuracy and computational load, resulting in low computational efficiency for recognition and detection.

[0078] To address this technical problem, this embodiment sets up multiple single-task modules and corresponding feature extraction modules in the attribute analysis model, forming the attribute analysis model in a modular manner. By combining different task modules and selecting the target task module for attribute analysis processing according to the detection task, the face attribute detection task becomes more diversified and can be flexibly completed. Each task module only needs to complete a single task, effectively reducing the consumption of computing resources and improving the model's recognition efficiency.

[0079] Step 140: Attribute analysis and extraction are performed on the aligned face image by each of the target task modules to obtain the single attribute analysis result set corresponding to the detection task; and feature extraction is performed on the aligned face image by each feature extraction module in the backbone model to obtain the face feature set.

[0080] In the specific implementation, each target task module can perform attribute analysis and extraction based on the input aligned face image to obtain the single attribute analysis result. The single attribute analysis results are aggregated to obtain the single attribute analysis result set. The single attribute analysis result is the prediction result obtained by the target task module based on the input data through single attribute analysis.

[0081] Similar to attribute analysis and extraction, since there is a one-to-one correspondence between the feature extraction module and the target task module, at least one feature extraction module is selected in this embodiment. This embodiment utilizes the feature extraction module corresponding to the target task module to analyze and extract facial features from the aligned face image, obtaining facial features. Each feature extraction module performs feature extraction based on the input data, thereby obtaining a facial feature set. Depending on the correspondence between the feature extraction module and the target task module, the facial features and the results of single attribute analysis can also have a one-to-one correspondence.

[0082] Therefore, in this embodiment, each selected target task module only needs to process the corresponding single detection task, while unselected task modules do not need to perform single detection tasks, or only need to perform data propagation during subsequent gradient propagation. This achieves a reasonable allocation of computing resources, effectively reduces resource consumption, and enables the selection and combination of required detection items in a modular manner, flexibly completing diverse face attribute detection tasks.

[0083] In an optional embodiment, this application embodiment performs attribute analysis and extraction on the aligned face image through each of the target task modules to obtain a single attribute analysis result set corresponding to the detection task, and performs feature extraction on the aligned face image through each feature extraction module in the backbone model to obtain a face feature set. Specifically, it may include: determining a first arrangement structure of each module in the attribute structure analysis model, wherein the first arrangement structure is the arrangement order between each of the target task modules, and each target task module corresponds one-to-one with each feature extraction module; performing attribute analysis and extraction on the aligned face image sequentially through the target task modules according to the first arrangement structure to obtain a single attribute analysis result set, and sequentially using the feature extraction modules to perform feature recognition and extraction on the aligned face image to obtain a face feature set.

[0084] In the specific implementation, there is a certain structural arrangement between the feature extraction modules and the task modules. This implementation can process the aligned face image input feature module and the target task module according to the arrangement structure between the modules. At this time, there can be at least two input processing methods, including but not limited to: sequential input processing and parallel input processing.

[0085] Specifically, for the sequential execution processing method, this embodiment can follow a sorted structure. First, the aligned face image is input into the target task module and the corresponding feature extraction module, which are ranked first in the sorted structure. At this time, the target task module and the corresponding feature extraction module can process the aligned face image respectively to obtain the corresponding prediction results. Then, the prediction results including attribute analysis, the prediction results of feature extraction, and the aligned face image output by the two modules are input into the target task module and the feature extraction module, which are ranked second in the sorted structure, for processing. This process continues, and after each person module and feature extraction module completes its current detection task according to the sorted structure, the prediction results and image data are input into the next module for processing until the detection task is completed, realizing gradient propagation data, i.e., executing step 150. After each target task module and feature extraction module completes its corresponding detection task, the next module performing the detection task can obtain the face features output by the previous module, thereby aggregating to obtain the single attribute analysis result set and the face feature set.

[0086] For parallel execution processing, in this embodiment of the application, the aligned face images can be directly input into each target task module and the corresponding feature extraction module in the sorted order. Through parallel execution, each target task module and the corresponding feature extraction module directly perform feature extraction processing on the aligned face images to obtain a single attribute analysis result set and a face feature set. Subsequently, each module performs gradient propagation on the extracted face features, i.e., executes step 150.

[0087] Step 150: Perform gradient propagation based on the single attribute analysis result set and the face feature set to obtain the face attribute analysis result corresponding to the face image to be analyzed.

[0088] In the specific implementation, gradient propagation can be understood as forward propagation. In this embodiment, the single attribute analysis result extracted by each target task module and the face features extracted by the corresponding feature extraction module are propagated forward in the forward propagation manner. Finally, the target task module that performs the detection task completes the attribute analysis, and the final prediction result of the face attribute analysis can be obtained. After the feature extraction module that performs the detection task completes the feature extraction, the final prediction result of the feature extraction can be obtained. By combining the two final prediction results and performing splicing / merging and other processing, the face attribute analysis result corresponding to the face image to be analyzed can be obtained.

[0089] In one optional embodiment, this application embodiment performs gradient propagation based on the single attribute analysis result set and the face feature set to obtain the face attribute analysis result corresponding to the face image to be analyzed. Specifically, it may include: according to the arrangement structure, using the single attribute analysis results in the single attribute analysis result set and the face features in the face feature set to perform forward propagation to obtain the face attribute analysis result.

[0090] As can be seen, this embodiment acquires a face image to be analyzed, performs face detection alignment on the face image according to a preset detection task to obtain an aligned face image, and then selects at least one corresponding target task module from a preset attribute structure analysis model according to the detection task. The aligned face image is input into the target task module and the backbone model in the attribute structure analysis model. Subsequently, attribute analysis and extraction are performed on the aligned face image by each target task module to obtain a single attribute analysis result set corresponding to the detection task, and feature extraction is performed on the aligned face image by each feature extraction module in the backbone model to obtain a face feature set. Gradient propagation is performed based on the single attribute analysis result set and the face feature set to obtain the face attribute analysis result corresponding to the face image to be analyzed. Therefore, the face attribute structure analysis method provided in this embodiment uses a single task module to perform a single detection task, combined with gradient propagation, to obtain the face attribute analysis result, and innovatively integrates LoRA fine-tuning technology and a fine-grained classification model. Since each task module runs independently, it only needs to execute a single detection task, without having to learn multiple complex tasks simultaneously within a limited parameter space. This allows it to be widely applied in multiple fields such as security monitoring, personalized recommendation, and human-computer interaction, achieving a balance between detection accuracy and computational efficiency. This solves the problems of high computational resource consumption, insufficient detection accuracy, and low detection efficiency in existing face attribute analysis methods.

[0091] Reference Figure 4This diagram illustrates a step-by-step flowchart of a modular face attribute structure analysis method model training according to an optional embodiment of this application. In a specific implementation, the modular face attribute structure analysis method provided in this embodiment may include the following steps during the model training phase:

[0092] Step 410: Obtain the face attribute dataset and build the initial analysis model.

[0093] The initial analysis model includes a single-task model and a backbone model. The single-task model includes task modules that correspond one-to-one with each feature extraction module in the backbone model. The feature extraction modules and the task modules call each other.

[0094] In this embodiment, the face attribute dataset may include, but is not limited to, face images and corresponding annotation data (also known as ground truth labels). In this embodiment, the feature extraction modules and task modules can be independent model modules. Task modules can correspond one-to-one with feature modules and can call each other.

[0095] For example, a single-task model can use the LoRA model, and the backbone model can use the ResNet50 model as the feature extraction model. Figure 5 As shown, a corresponding task module, LoRA layer, also known as the task layer, can be constructed in the LoRA model according to the detection task. The task module corresponds to the feature extraction module. Figure 5 The ResNet model can be mainly composed of residual stages, that is, the feature extraction module can also be mainly composed of residual feature extraction module and input module. In order to facilitate modular calling with LoRA, the Input stem, stage 1, stage 2... stage N model of the ResNet model are split and packaged into independent model modules. The training process is still arranged according to the structure.

[0096] Step 420: Based on the preset background feature key points, perform face detection and face alignment on the sample face images in the face attribute dataset to obtain aligned face image samples.

[0097] For a description of face detection, face alignment, and background feature key points, please refer to step 120 above. This embodiment will not elaborate on these details.

[0098] Step 430: Select a task module corresponding to the preset attribute detection item from the single-task model as the task module to be trained, and input the aligned face image sample into the task module to be trained and the backbone model.

[0099] In this embodiment, the training direction corresponding to each task module can be predetermined, that is, it can be determined which attribute analysis detection task the task module is used to identify and detect. Then, according to the detection task, the aligned face image samples and the task modules in the single-task model are divided, the task module to be trained is selected, and the aligned face image samples are input into the corresponding task module to be trained for model training. In addition, the aligned face image samples can be simultaneously input into the feature extraction module corresponding to the task module to be trained for feature extraction.

[0100] In a specific implementation, before training the model for each task module in the single-task model, this embodiment can also pre-train the model for each feature extraction module of the backbone model. The training process for each feature extraction module in the backbone model can be as follows: Figure 5 The arrangement structure shown can be used for sequential or parallel feature extraction training.

[0101] In an optional embodiment, before inputting aligned face image samples into the training task module and the backbone model, this embodiment may further include: acquiring a face recognition dataset; using the face recognition dataset to train each feature extraction module in the backbone model for feature extraction, and using a preset loss function to optimize the parameters of the feature extraction results output by the feature extraction modules until the loss converges, thereby obtaining a trained backbone model; wherein, after the backbone model is pre-trained, the parameters are fixed, and the backbone model does not participate in the attribute analysis training of the training task module.

[0102] In this embodiment, the ResNet model is first trained using a face recognition dataset to ensure that the low-level operators learn complete facial detail features. This example uses the publicly available dataset Glint360K as the face recognition dataset to train the ResNet model, using Cosfaceloss as the loss function, and training continues until the loss converges. In this embodiment, the ResNet model is primarily trained using face recognition methods to ensure that the convolutional operators learn sufficient facial feature information. The ResNet model is trained only once in this scenario, and subsequent training with fixed parameters does not participate in LoRA training.

[0103] Step 440: Use the aligned face image samples to perform attribute analysis training on the task module to be trained, obtain the single prediction result output by the task module to be trained, and obtain the feature prediction result output by the backbone model for the aligned face image samples.

[0104] In its implementation, LoRA is designed with a corresponding number of layers according to the feature extraction model structure. Through in-depth exploration, LoRA modules can be designed for each linear layer within the module. By training the corresponding LoRA separately using aligned face image samples, the single-item prediction results output by LoRA during attribute analysis training can be obtained, and the feature prediction results output by the corresponding feature extraction module can also be acquired.

[0105] Optionally, in embodiments of this application where the task module to be trained includes a first matrix and a second matrix, the above-mentioned attribute analysis training of the task module to be trained using the aligned face image samples may specifically include: performing matrix multiplication using the first matrix and the second matrix to obtain the model update bias; performing attribute analysis training based on the model update bias, the aligned face image samples, and the initial parameters of the model to obtain the single prediction result output by the task module to be trained.

[0106] In one optional embodiment, this application embodiment performs attribute analysis training based on the model update bias, the aligned face image samples, and the initial parameters of the model, specifically including: according to Determine the individual prediction results; among them, Here, f represents the model's predicted value, and f is the model function. For the input aligned face image sample, For model parameters, These are the parameters of the original, untrained model. The parameters were varied to train the model and fit them using two low-rank matrices. and Multiplication means, These are the parameters for the forward propagation of the feature extraction module in the backbone model. These are the parameters for the forward propagation of the task module to be trained.

[0107] In this implementation, to ensure the model can accurately analyze the attributes of the input image, LoRA in this example can consist of two trainable matrices, denoted as matrix A and matrix B. Matrix A can be the first matrix, and matrix B can be the second matrix; this example does not impose any restrictions. Multiplying matrices A and B yields the deviation in model parameter updates. Therefore, the shape of matrix A is the same as the input size, such as matrix A = (m, rank), and the size of matrix B is the same as the output size, such as matrix B = (rank, n). During model training, the backbone model parameters can be frozen and not used in training. Simultaneously, during weight initialization, matrix A is initialized with Gaussian noise, while matrix B is initialized with a zero matrix, ensuring training starts from the bottom model. Because AB is a zero matrix, LoRA is initially inactive. During model training, assuming X is used as input data, the output length after inputting into the model matches the number of categories. , The model prediction values ​​are obtained by each module through attribute analysis and feature extraction based on the input data. In this embodiment, the A and B matrices provide a benchmark for optimizing the model parameters, so that the predicted results of the model input can be close to the true labels corresponding to the samples, thereby improving the model's ability to analyze the input images.

[0108] In this embodiment, it is only necessary to train the corresponding LoRA layer for each stage in the pre-trained ResNet50 for face recognition, and then input and add them in the calculation process to obtain the prediction result of the trained model and efficiently utilize the feature extraction capability of the backbone network.

[0109] Step 450: Perform gradient propagation based on the single prediction result and the feature prediction result to obtain the model prediction result.

[0110] Reference Figure 4 As shown, the feature extraction module and LoRA module of each layer predict the corresponding result from the input data, and then input the prediction result and the input data into the next layer for forward propagation. After the feature extraction module and LoRA module of the last layer complete the prediction, the prediction results are merged to obtain the final model prediction result.

[0111] In an optional embodiment, this application embodiment performs gradient propagation based on the single-item prediction result and the feature prediction result to obtain the model prediction result, which may specifically include:

[0112] according to Gradient propagation is performed; the backbone model includes an input feature extraction module and at least four residual feature extraction modules, while the single-task model includes at least five training task modules corresponding to the feature extraction modules of the backbone model. , , , as well as All of these are forward propagations of the feature extraction modules of the backbone model. , , , as well as All of these are forward propagations of each training task module in a single-task model. This represents the model's prediction results.

[0113] In this embodiment, the ResNet model consists of four residual stages, assuming that the forward propagation of each stage is equivalent to the function , , , , Forward propagation in the LoRA model is equivalent to the function , , , , Input data X into the overall model is equivalent to inputting data separately. and Due to the ResNet model The parameters are fixed and do not participate in parameter updates, so what is actually being trained is the LoRA part. The parameters are thus used to form a model with ResNet as the fixed core and LoRA trained on different data as modules. By merging the outputs after inputting the same data, the prediction results of the corresponding attributes can be obtained.

[0114] Step 460: Calculate the loss using the real labels corresponding to the sample face images and the model prediction results, and optimize the parameters of the task module to be trained based on the loss calculation results until the single-task model training is completed, thus obtaining the trained attribute structure analysis model.

[0115] In this embodiment, after obtaining the prediction results output by the model, the model prediction results and the real labels can be used to calculate the loss, such as cross-entropy, focal loss, etc., and backpropagation can be used to minimize the deviation between logits and the real labels. The LoRA parameters are continuously optimized until training is completed and a trained attribute structure analysis model is obtained.

[0116] In the specific implementation, after the attribute structure analysis model is trained, the attribute structure analysis model can be used to perform attribute analysis and feature extraction on the aligned face image corresponding to the face image to be analyzed according to the selected detection task. The data output by each layer module is merged using the forward propagation method to finally obtain the face attribute analysis result corresponding to the face image to be analyzed.

[0117] In summary, in the model training phase of the attribute structure analysis model, this application acquires a face attribute dataset and constructs an initial analysis model containing a single-task model and a backbone model. Then, based on preset background feature key points, face detection and alignment are performed on sample face images from the face attribute dataset to obtain aligned face image samples. A task module corresponding to a preset attribute detection item is selected from the single-task model as a training task module. The aligned face image samples are input into the training task module and the backbone model. Attribute analysis training is performed on the training task module using the aligned face image samples to obtain the single-item prediction result output by the training task module. The feature prediction result output by the backbone model for the aligned face image samples is also obtained. Gradient propagation is performed based on the single-item prediction result and the feature prediction result to obtain the model prediction result. Loss calculation is performed using the real labels corresponding to the sample face images and the model prediction result. The parameters of the training task module are optimized based on the loss calculation result until the single-task model training is completed, resulting in a trained attribute structure analysis model. As can be seen, this embodiment sets up multiple task modules and feature extraction modules corresponding to detection tasks in the attribute analysis model. During model training, only single-task training is needed for the detection task corresponding to the task module, eliminating the need for full-task training. This effectively reduces the model learning cost, avoids overfitting and underlearning, reduces computational resource requirements, and ensures detection and recognition accuracy. Thus, it finds the optimal balance between accuracy and computational load. The customizable module options further meet different business needs without redundant detection items. In practical applications, this application can flexibly complete diverse face attribute detection tasks by combining with different task modules. It allows for the modular selection and combination of required detection items according to different business needs, thereby achieving efficient and customized face attribute analysis, improving model recognition accuracy, and solving the problems existing in current face attribute analysis methods that use multi-attribute classification models or multiple single-attribute classification models for detection.

[0118] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should know that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps may be performed in other orders or simultaneously.

[0119] like Figure 6 As shown in the figure, this application embodiment also provides a modular face attribute structure analysis device 600, including:

[0120] Image acquisition module 610 is used to acquire the face image to be analyzed;

[0121] The face detection and alignment module 620 is used to perform face detection and alignment on the face image to be analyzed according to a preset detection task, so as to obtain an aligned face image.

[0122] The input module 630 is used to select at least one target task module from a preset attribute structure analysis model according to the detection task, and input the aligned face image into the target task module and the backbone model in the attribute structure analysis model;

[0123] The feature extraction module 640 is used to perform attribute analysis and extraction on the aligned face image through each of the target task modules to obtain a single attribute analysis result set corresponding to the detection task, and to perform feature extraction on the aligned face image through each feature extraction module in the backbone model to obtain a face feature set.

[0124] The gradient propagation module 650 is used to perform gradient propagation based on the single attribute analysis result set and the face feature set to obtain the face attribute analysis result corresponding to the face image to be analyzed.

[0125] Optionally, the face detection and alignment module 620 includes:

[0126] The detection task and key point acquisition submodule is used to acquire the preset detection task and the preset background feature key points corresponding to the detection task;

[0127] The face detection submodule is used to perform face detection on the face image to be analyzed, and select key alignment positions from the face detection results according to the detection task.

[0128] The alignment submodule is used to align the face image to be analyzed using the background feature key points and the key alignment positions to obtain an aligned face image; wherein, the background feature key points are key points including objects around the face, and the aligned face image includes face features and face background features.

[0129] Optionally, the feature extraction module 640 includes:

[0130] The permutation structure determination module is used to determine the first permutation structure of each module in the attribute structure analysis model. The first permutation structure is the arrangement order between the target task modules, and the target task modules correspond one-to-one with the feature extraction modules.

[0131] The extraction submodule is used to sequentially perform attribute analysis and extraction on the aligned face image through the target task module according to the first arrangement structure to obtain a single attribute analysis result set, and sequentially use the feature extraction module to perform feature recognition and extraction on the aligned face image to obtain a face feature set.

[0132] Optionally, the gradient propagation module 650 includes:

[0133] The forward propagation submodule is used to perform forward propagation according to the arrangement structure, using the single attribute analysis results in the single attribute analysis result set and the facial features in the facial feature set, to obtain the facial attribute analysis results.

[0134] Optionally, the modular face attribute structure analysis device 600 further includes:

[0135] The attribute sample acquisition and model building module is used to acquire a face attribute dataset and build an initial analysis model. The initial analysis model includes a single-task model and a backbone model. The single-task model includes task modules that correspond one-to-one with each feature extraction module in the backbone model. The feature extraction modules and the task modules call each other.

[0136] The sample face detection and alignment module is used to perform face detection and face alignment based on the sample face images in the face attribute dataset according to preset background feature key points, so as to obtain aligned face image samples.

[0137] The sample input module is used to select a task module corresponding to a preset attribute detection item from the single-task model as a task module to be trained, and input the aligned face image sample into the task module to be trained and the backbone model.

[0138] The training module is used to perform attribute analysis training on the task module to be trained using the aligned face image samples, to obtain the single prediction result output by the task module to be trained, and to obtain the feature prediction result output by the backbone model for the aligned face image samples.

[0139] The prediction result gradient propagation module is used to perform gradient propagation based on the single prediction result and the feature prediction result to obtain the model prediction result;

[0140] The parameter optimization module is used to calculate the loss using the real labels corresponding to the sample face images and the model prediction results, and to optimize the parameters of the task module to be trained based on the loss calculation results until the single-task model is trained and a trained attribute structure analysis model is obtained.

[0141] Optionally, the modular face attribute structure analysis device 600 further includes:

[0142] The face recognition sample acquisition module is used to acquire face recognition datasets;

[0143] The feature extraction training module is used to train each feature extraction module in the backbone model using the face recognition dataset, and to optimize the parameters of the feature extraction results output by the feature extraction module using a preset loss function until the loss converges, thus obtaining the trained backbone model; wherein, the parameters of the backbone model are fixed after the backbone model is pre-trained, and the backbone model does not participate in the attribute analysis training of the task module to be trained.

[0144] Optionally, the task module to be trained includes a first matrix and a second matrix, and the training module includes:

[0145] The model update bias acquisition submodule is used to perform matrix multiplication using the first matrix and the second matrix to obtain the model update bias.

[0146] The attribute analysis training submodule is used to perform attribute analysis training based on the model update bias, the aligned face image samples, and the initial parameters of the model, so as to obtain the single prediction result output by the task module to be trained.

[0147] Optionally, the attribute analysis training submodule is specifically used for: based on Determine the individual prediction results; among them, Here, f represents the model's predicted value, and f is the model function. For the input aligned face image sample, For model parameters, These are the parameters of the original, untrained model. The parameters were varied to train the model and fit them using two low-rank matrices. and Multiplication means, These are the parameters for the forward propagation of the feature extraction module in the backbone model. These are the parameters for the forward propagation of the task module to be trained.

[0148] Optionally, the gradient propagation module for the prediction result is specifically used for:

[0149] according to Gradient propagation is performed; the backbone model includes an input feature extraction module and at least four residual feature extraction modules, while the single-task model includes at least five training task modules corresponding to the feature extraction modules of the backbone model. , , , as well as All of these are forward propagations of the feature extraction modules of the backbone model. , , , as well as All of these are forward propagations of each training task module in a single-task model. This represents the model's prediction results.

[0150] It should be noted that the modular face attribute structure analysis device provided in the embodiments of this application can execute the modular face attribute structure analysis method provided in any embodiment of this application, and has the corresponding functions and beneficial effects of the execution method.

[0151] In practical implementation, the aforementioned modular face attribute structure analysis device can be integrated into the device. This allows the device to select task modules from the attribute analysis model to perform attribute analysis and feature extraction on face images based on customized detection tasks. As an electronic device, it can flexibly complete diverse face attribute detection tasks, thereby achieving efficient face attribute analysis and improving model recognition accuracy. This electronic device can consist of two or more physical entities, or it can consist of a single physical entity. For example, the electronic device can be a personal computer (PC), a computer, a server, etc. This application embodiment does not impose specific limitations in this regard.

[0152] like Figure 7 As shown, this application embodiment provides an electronic device, including a processor 111, a communication interface 112, a memory 113, and a communication bus 114. The processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. The memory 113 is used to store computer programs. When the processor 111 executes the program stored in the memory 113, it implements the steps of the modular face attribute structure analysis method provided in any of the aforementioned method embodiments. For example, the modular face attribute structure analysis method may include the following steps: acquiring a face image to be analyzed; performing face detection alignment on the face image to be analyzed according to a preset detection task to obtain an aligned face image; selecting at least one corresponding target task module from a preset attribute structure analysis model according to the detection task, and inputting the aligned face image into the target task module and the backbone model in the attribute structure analysis model; performing attribute analysis and extraction on the aligned face image through each target task module to obtain a single attribute analysis result set corresponding to the detection task, and performing feature extraction on the aligned face image through each feature extraction module in the backbone model to obtain a face feature set; performing gradient propagation based on the single attribute analysis result set and the face feature set to obtain the face attribute analysis result corresponding to the face image to be analyzed.

[0153] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the modular face attribute structure analysis method provided in any of the foregoing method embodiments.

[0154] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0155] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A modular method for analyzing facial attribute structure, characterized in that, include: Obtain the face image to be analyzed; Based on the preset detection task, key points for face alignment are selected to perform face detection and alignment on the face image to be analyzed, thereby obtaining an aligned face image. According to the detection task, at least one target task module is selected from the preset attribute structure analysis model, and the aligned face image is input into the target task module and the backbone model in the attribute structure analysis model. The aligned face image is analyzed and extracted by each of the target task modules to obtain a set of single attribute analysis results corresponding to the detection task. The aligned face image is also extracted by each feature extraction module in the backbone model to obtain a face feature set. The target task module and the feature extraction module correspond one-to-one. Gradient propagation is performed based on the single attribute analysis result set and the face feature set to obtain the face attribute analysis result corresponding to the face image to be analyzed.

2. The method according to claim 1, characterized in that, The step of selecting key points for face alignment according to a preset detection task and performing face detection and alignment on the face image to be analyzed to obtain an aligned face image includes: Obtain the preset detection task and the preset background feature key points corresponding to the detection task; Face detection is performed on the face image to be analyzed, and key alignment positions are selected from the face detection results according to the detection task. By using the background feature key points and combining the key alignment positions, the face image to be analyzed is aligned to obtain an aligned face image. The background feature key points are key points containing objects around the face, and the aligned face image includes face features and face background features.

3. The method according to claim 1, characterized in that, The step involves performing attribute analysis and extraction on the aligned face image through each of the target task modules to obtain a single attribute analysis result set corresponding to the detection task, and performing feature extraction on the aligned face image through each feature extraction module in the backbone model to obtain a face feature set, including: Determine the first arrangement structure of each module in the attribute structure analysis model, wherein the first arrangement structure is the order of the target task modules; According to the first arrangement structure, the aligned face image is sequentially analyzed and extracted by the target task module to obtain a single attribute analysis result set, and then the aligned face image is sequentially identified and extracted by the feature extraction module to obtain a face feature set.

4. The method according to claim 3, characterized in that, The step of performing gradient propagation based on the single-attribute analysis result set and the face feature set to obtain the face attribute analysis result corresponding to the face image to be analyzed includes: According to the arrangement structure, the face attribute analysis results are obtained by forward propagation using the single attribute analysis results in the single attribute analysis result set and the face features in the face feature set.

5. The method according to claim 1, characterized in that, Before selecting at least one corresponding target task module from the preset attribute structure analysis model, the method further includes: Acquire a facial attribute dataset and construct an initial analysis model. The initial analysis model includes a single-task model and a backbone model. The single-task model includes task modules that correspond one-to-one with each feature extraction module in the backbone model. The feature extraction modules and the task modules call each other. Based on the preset background feature key points, face detection and face alignment are performed on the sample face images in the face attribute dataset to obtain aligned face image samples. Select a task module corresponding to a preset attribute detection item from the single-task model as a task module to be trained, and input the aligned face image sample into the task module to be trained and the backbone model; The alignment face image samples are used to train the task module to be trained by attribute analysis, and the single prediction result output by the task module to be trained is obtained. The feature prediction result output by the backbone model for the alignment face image samples is also obtained. Gradient propagation is performed based on the individual prediction results and the feature prediction results to obtain the model prediction results; The loss is calculated using the real labels corresponding to the sample face images and the model prediction results. The parameters of the task module to be trained are optimized based on the loss calculation results until the single-task model is trained, resulting in a trained attribute structure analysis model.

6. The method according to claim 5, characterized in that, Before inputting the aligned face image samples into the training task module and the backbone model, the method further includes: Obtain the face recognition dataset; The feature extraction modules in the backbone model are trained using the face recognition dataset, and the parameters of the feature extraction results output by the feature extraction modules are optimized using a preset loss function until the loss converges, thus obtaining the trained backbone model. The parameters of the backbone model are fixed after pre-training, and the backbone model does not participate in the attribute analysis training of the task module to be trained.

7. The method according to claim 5, characterized in that, The task module to be trained includes a first matrix and a second matrix. The step of training the task module using the aligned face image samples through attribute analysis includes: The model update bias is obtained by multiplying the first matrix and the second matrix. Based on the model update bias, the aligned face image samples, and the initial parameters of the model, attribute analysis training is performed to obtain the single prediction result output by the task module to be trained.

8. The method according to claim 7, characterized in that, The attribute analysis training based on the model update bias, the aligned face image samples, and the initial model parameters includes: according to Determine the individual prediction results; in, Here, f represents the model's predicted value, and f is the model function. For the input aligned face image sample, For model parameters, These are the parameters of the original, untrained model. The parameters were varied to train the model and fit them using two low-rank matrices. and Multiplication means, These are the parameters for the forward propagation of the feature extraction module in the backbone model. These are the parameters for the forward propagation of the task module to be trained.

9. The method according to claim 5, characterized in that, The step of performing gradient propagation based on the individual prediction result and the feature prediction result to obtain the model prediction result includes: according to Perform gradient propagation; The backbone model includes an input feature extraction module and at least four residual feature extraction modules, while the single-task model includes at least five training task modules corresponding to the feature extraction modules of the backbone model. , , , as well as All of these are forward propagations of the feature extraction modules of the backbone model. , , , as well as All of these are forward propagations of each training task module in a single-task model. This represents the model's prediction results.

10. A modular facial attribute structure analysis device, characterized in that, include: The image acquisition module is used to acquire the face image to be analyzed; The face detection and alignment module is used to select key points for face alignment according to a preset detection task and perform face detection and alignment on the face image to be analyzed to obtain an aligned face image. The input module is used to select at least one target task module from the preset attribute structure analysis model according to the detection task, and input the aligned face image into the target task module and the backbone model in the attribute structure analysis model; The feature extraction module is used to perform attribute analysis and extraction on the aligned face image through each of the target task modules to obtain a single attribute analysis result set corresponding to the detection task, and to perform feature extraction on the aligned face image through each feature extraction module in the backbone model to obtain a face feature set. The target task module and the feature extraction module correspond one-to-one. The gradient propagation module is used to perform gradient propagation based on the single attribute analysis result set and the face feature set to obtain the face attribute analysis result corresponding to the face image to be analyzed.

11. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps of the modular face attribute structure analysis method according to any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the modular face attribute structure analysis method as described in any one of claims 1-9.

Images