Face attribute recognition method, terminal and storage medium

By designing an initial face attribute recognition model containing shared layers and branches, training and fusion of global and local models separately, the problems of slow face attribute recognition speed and low accuracy in the existing technology are solved, and the rapid and high-precision multi-dimensional Attribute recognition.

CN113963417BActive Publication Date: 2024-11-05SHENZHEN MAXVISION TECH
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202111312411.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-08
Publication Date
2024-11-05
Estimated Expiration
2041-11-08

AI Technical Summary

Technical Problem

The existing facial attribute recognition methods based on deep learning have problems such as slow recognition speed and low recognition accuracy, especially when multi-branch models are difficult to achieve high accuracy under data imbalance.

Method used

The initial face attribute recognition model design includes a sharing layer, a global face attribute branch and a local face attribute branch. By training the global and local models separately and integrating them, the optimal face attribute recognition model is finally generated, avoiding the complexity of multi-attribute labeling. Effects of sexual and data imbalance on inter-branches.

Benefits of technology

It realizes a fast time to recognize multiple attributes of faces at the same time, reduces the use of memory resources, and improves the recognition rate and accuracy of face attributes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113963417B_ABST
    Figure CN113963417B_ABST
Patent Text Reader

Abstract

The present application provides a facial attribute recognition method, a terminal and a storage medium. The facial attribute recognition method comprises: obtaining a facial image; classifying and preprocessing the facial image according to global facial attributes and multiple local facial attributes; designing an initial facial attribute recognition model, inputting the global facial attribute training data set into the global facial attribute recognition model for training, and obtaining an optimal global facial attribute recognition model; inputting multiple local facial attribute training data sets into the local facial attribute recognition model respectively, and loading the parameters of the optimal global facial attribute recognition model for training, and obtaining multiple optimal local facial attribute recognition models; fusing the optimal global facial attribute recognition model and multiple optimal local facial attribute recognition models to obtain a final facial attribute recognition model; the facial attribute recognition method of the present application has the characteristics of fast recognition speed and high recognition accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of biometric identification technology, and more specifically, to a facial attribute recognition method, terminal and storage medium. Background Art

[0002] With the continuous development of Internet technology, the integration of security and AI technology is becoming increasingly close. Among them, facial attribute recognition is not only used in access control systems to assist facial recognition, but also in personnel identification in other scenarios. For example, in smart communities, based on facial images, gender, glasses, masks, hats and other attributes are identified to monitor and track outsiders in the community, etc., to improve the response capabilities of security work.

[0003] There are two main methods for facial attribute recognition based on deep learning. One is to use multiple attribute recognition models for comprehensive recognition, but multiple models have the problem of occupying large resources and slow operation. The other is to use a multi-branch recognition model to recognize all facial attributes at the same time. Since this model needs to label each attribute of the image, it requires a lot of labeling work, and the data of each attribute is mostly unbalanced, so it is difficult for the model to achieve high recognition accuracy on multiple branches at the same time. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a facial attribute recognition method, a terminal and a storage medium to solve the technical problems of slow recognition speed and low recognition accuracy in the facial attribute recognition process in the prior art.

[0005] To achieve the above purpose, the technical solution adopted by this application is: to provide a face attribute recognition method, including:

[0006] The monitoring system collects images and obtains facial images;

[0007] Classifying and preprocessing the face images according to global face attributes and multiple local face attributes to generate a global face attribute training data set and multiple local face attribute data sets;

[0008] Designing an initial face attribute recognition model, the initial face attribute recognition model comprising a shared layer, a global face attribute branch, and a plurality of local face attribute branches, saving a global face attribute recognition model composed of the shared layer and the global face attribute branches, and saving a local face attribute recognition model composed of the shared layer and the local face attribute branches;

[0009] Inputting the global face attribute training data set into the global face attribute recognition model for training to obtain an optimal global face attribute recognition model; inputting the plurality of local face attribute training data sets into the local face attribute recognition models respectively, and loading the parameters of the optimal global face attribute recognition model for training to obtain a plurality of optimal local face attribute recognition models;

[0010] The optimal global face attribute recognition model and the multiple optimal local face attribute recognition models are merged to obtain a final face attribute recognition model;

[0011] Input the test set into the final face attribute recognition model to obtain the corresponding face attribute recognition results.

[0012] Preferably, the global face attributes include gender attributes.

[0013] Preferably, the local facial attributes include at least one of glasses, a mask and a hat.

[0014] Preferably, the pre-processing step comprises:

[0015] Performing enhancement processing on the facial image;

[0016] The enhanced face image is converted into a model input size.

[0017] Preferably, the initial face attribute recognition model uses the EfficientNet model as a benchmark model, the first N layers of the initial face attribute recognition model are the shared layers, the Nth layer of the initial face attribute recognition model is a node of a multi-branch model, and the Nth layer is followed by a global face attribute branch and a variety of local face attribute branches.

[0018] Preferably, the step of inputting the global face attribute training data set into the global face attribute recognition model for training comprises:

[0019] Set network parameters;

[0020] By continuously reducing the cross entropy loss until the model converges;

[0021] The loss function of the network is:

[0022] Among them, m represents the number of samples, n represents the number of categories, P is a function with a value of 1 or 0, which means that if the true category of sample i is equal to category j, it takes 1, otherwise it takes 0, and q represents the predicted probability that sample i belongs to category j, and its value range is 0 to 1.

[0023] Preferably, the step of loading the parameters of the optimal global face attribute recognition model comprises:

[0024] Obtaining parameters of a shared layer in the optimal global face attribute recognition model;

[0025] Freezing parameters of a shared layer in the local face attribute recognition model;

[0026] The network parameters of the local face attribute branch are set for training.

[0027] Preferably, the monitoring system performs image acquisition, and after acquiring the face image, further comprises the steps of:

[0028] The facial images are divided into a training set and a test set in a ratio of 9:1, wherein the training set is used for image classification and preprocessing, and the test set is used to input the test results into the final facial attribute recognition model.

[0029] The present application also provides a terminal, which includes a facial attribute recognition model, a memory, a processor, and a recognition program stored in the memory and executable on the processor, wherein the facial attribute recognition model is connected to the processor, and the recognition program implements the steps of the facial attribute recognition method described above when executed by the processor.

[0030] The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the facial attribute recognition method described above are implemented.

[0031] Compared with the prior art, the face attribute recognition method provided by the present application can simultaneously recognize multiple attributes of the face during the face attribute recognition process by setting a global face attribute branch and multiple local face attribute branches, thereby shortening the time for the model to recognize attributes, reducing the memory resources occupied, and improving the recognition rate of various face attributes; the global face attribute recognition model and the local face attribute recognition model are trained separately, and then merged into the final face attribute recognition model, which not only avoids the complex work of multi-attribute labeling, but also alleviates the mutual influence of data imbalance on branches during multi-branch model training, thereby improving the recognition accuracy of each branch.

[0032] The beneficial effects of the face attribute recognition terminal and storage medium provided by the present application are: by adopting the above-mentioned face attribute recognition method, multiple attributes of the face can be recognized at the same time, the time for the model to recognize attributes can be shortened, the memory resources occupied can be reduced, and the recognition rate of various attributes of the face can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0034] Figure 1 A schematic diagram of a process for face attribute recognition provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of an initial face attribute recognition model provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of the training process of the optimal global face attribute recognition model provided in an embodiment of the present application;

[0037] Figure 4 A schematic diagram of the training process of the optimal local face attribute recognition model provided in an embodiment of the present application;

[0038] Figure 5 A schematic diagram of the process of obtaining the final face attribute recognition model by fusion provided in the embodiment of the present application;

[0039] Figure 6 A schematic diagram of a facial attribute recognition terminal and a storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0042] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0043] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0044] Please also read Figures 1 to 5 Now, the face attribute recognition method provided by the embodiment of the present application is described. The face attribute recognition method comprises:

[0045] Step S1, the monitoring system performs image acquisition to obtain a face image.

[0046] Step S2, classifying and preprocessing the face image according to global face attributes and multiple local face attributes, to generate a global face attribute training data set and multiple local face attribute data sets.

[0047] Step S3, designing an initial face attribute recognition model, the initial face attribute recognition model includes a shared layer, a global face attribute branch and multiple local face attribute branches, saving the global face attribute recognition model composed of the shared layer and the global face attribute branches, and saving the local face attribute recognition model composed of the shared layer and the local face attribute branches.

[0048] Step S4, inputting the global face attribute training data set into the global face attribute recognition model for training to obtain the optimal global face attribute recognition model; inputting the multiple local face attribute training data sets into the local face attribute recognition model respectively, and loading the parameters of the optimal global face attribute recognition model for training to obtain multiple optimal local face attribute recognition models.

[0049] Step S5, fusing the optimal global face attribute recognition model and the multiple optimal local face attribute recognition models to obtain a final face attribute recognition model.

[0050] Step S6, input the test set into the final face attribute recognition model to obtain the corresponding face attribute recognition result.

[0051] It is worth noting that in step S1, the monitoring system performs image acquisition and can obtain face images according to the structure of the initial face attribute recognition model, so that the number of each image classification is relatively uniform. In step S4, since the global face attribute recognition model and the local face attribute recognition model are trained separately, the data do not affect each other during the training of the multi-branch model, ensuring that the generated global face attribute training data set and multiple local face attribute data sets are relatively balanced, thereby improving the recognition accuracy of each face attribute branch.

[0052] In addition, when data annotation is performed on facial images, a large amount of work of multi-attribute annotation can be avoided, and the attribute annotation areas do not overlap with each other.

[0053] In step S2, the global face attributes are opposite to the local face attributes. The global face attributes refer to the feature attributes that cover a relatively wide area in the face image, such as gender, skin, face shape, etc.; the local face attributes refer to the feature attributes that cover a relatively narrow area in the face image, such as glasses, masks, hats, etc.

[0054] In step S3, since the shared layer is shared by the global face attribute recognition model and the local face attribute recognition model, N can be set according to the actual application requirements, that is, the first N layers of the initial face attribute recognition model can be set as the shared layer, and the Nth layer of the initial face attribute recognition model is the node of the multi-branch model, and the Nth layer is followed by the global face attribute branch and multiple local face attribute branches. Since the global face attribute recognition model and the local face attribute recognition model are trained separately, the shared layer also provides a basis for the fusion in step S5.

[0055] In step S4, the global face attribute training data set is input into the global face attribute recognition model, and the network parameters are set for training. By continuously reducing the cross entropy loss until the model converges, multiple face attribute recognition models are usually obtained. At this time, the optimal global face attribute recognition model needs to be screened out; similarly, each local face attribute training data set is input into the local face attribute recognition model, and the parameters of the optimal global face attribute recognition model are loaded for training, and the shared layer of the local face attribute recognition model is frozen, and the network parameters are set for training until the set number of training times is reached, and the optimal local face attribute recognition model is obtained. Among them, "loading the parameters of the optimal global face attribute recognition model" refers to the parameters of the shared layer, which ensures that the optimal global face attribute recognition model and the optimal local face attribute recognition model can be fused through the shared layer with the same parameters in step S5.

[0056] In step S6, the input test set can verify the final face attribute recognition model. The face attribute recognition method sets a global face attribute branch and multiple local face attribute branches through a multi-branch residual deep neural network. In the face attribute recognition process, multiple attributes of the face can be recognized at the same time, thereby shortening the time for the model to recognize attributes, reducing the memory resources used, and improving the recognition rate of various face attributes.

[0057] Compared with the prior art, the face attribute recognition method provided by the present application can simultaneously recognize multiple attributes of the face during the face attribute recognition process by setting a global face attribute branch and multiple local face attribute branches, thereby shortening the time for the model to recognize attributes, reducing the memory resources occupied, and improving the recognition rate of various face attributes; the global face attribute recognition model and the local face attribute recognition model are trained separately, and then merged into the final face attribute recognition model, which not only avoids the complex work of multi-attribute labeling, but also alleviates the mutual influence of data imbalance on branches during multi-branch model training, thereby improving the recognition accuracy of each branch.

[0058] In another embodiment of the present application, in step S2, the global face attributes include gender attributes.

[0059] It is understandable that the gender in the gender training data set includes two characteristics, male and female. Since male and female cover a wide range of facial attributes, they have global characteristics. When training the local facial attribute recognition model, the parameters of the optimal global facial attribute recognition model can be loaded, which is conducive to further improving the fusion degree and recognition efficiency.

[0060] Further, in step S2, the local facial attributes include at least one of glasses, a mask and a hat.

[0061] It can be understood that the glasses, masks and hats are all recognizable features of facial attributes, and their characteristics are that they have local feature representation, which facilitates feature recognition and data labeling.

[0062] In another embodiment of the present application, in step S2, the preprocessing step includes:

[0063] Step S2.1, performing enhancement processing on the face image;

[0064] Step S2.2, converting the enhanced face image into a model input size.

[0065] It is understandable that the enhancement processing includes at least one of image scaling, flipping and rotation, affine transformation, and adjusting brightness and darkness. For example, the face image can be scaled only, or the face image can be scaled first and then affine transformed, or even the face image can be affine transformed first and then scaled. This can reduce the demand for basic data, and batches of data can be generated from a small amount of basic data to increase the diversity of training data.

[0066] In another embodiment of the present application, please refer to Figure 2 The initial face attribute recognition model uses the efficientnet model as the benchmark model, the first N layers of the initial face attribute recognition model are the shared layers, the Nth layer of the initial face attribute recognition model is a node of a multi-branch model, and the Nth layer is followed by a global face attribute branch and a variety of local face attribute branches.

[0067] It is understandable that N can be customized according to actual application needs. The EfficientNet model is a model obtained by Google through machine search. This model is a fast and high-precision model. It uses the depth (depth), width (width), and input image resolution (resolution) joint adjustment technology. EfficientNet can design a standardized convolutional network expansion method that can achieve high accuracy and fully save computing resources. The three dimensions of resolution, depth and width can be balanced to achieve optimization of the efficiency and accuracy of the capture network.

[0068] In another embodiment of the present application, please refer to Figure 3 In step S4, the step of inputting the global face attribute training data set into the global face attribute recognition model for training includes:

[0069] Step S4.1, setting network parameters;

[0070] Step S4.2, by continuously reducing the cross entropy loss until the model converges;

[0071] The loss function of the network is:

[0072] Among them, m represents the number of samples, n represents the number of categories, P is a function with a value of 1 or 0, which means that if the true category of sample i is equal to category j, it takes 1, otherwise it takes 0, and q represents the predicted probability that sample i belongs to category j, and its value range is 0 to 1.

[0073] It can be understood that in step S4.1, the network parameters include the network parameters of the shared layer and the network parameters of the local face attribute branch. The number of face attribute categories includes the sum of global face attributes and multiple local face attributes. The loss function can be used to optimize the model so as to obtain the optimal global face attribute recognition model.

[0074] For further information, please also refer to Figure 4 , the step of loading the parameters of the optimal global face attribute recognition model comprises:

[0075] Step S4.3, obtaining parameters of a shared layer in the optimal global face attribute recognition model;

[0076] Step S4.4, freezing the parameters of the shared layer in the local face attribute recognition model;

[0077] Step S4.5, setting the network parameters of the local face attribute branch for training.

[0078] It can be understood that since the global face attribute branch and the multiple local face attribute branches are branches behind the shared layer, the parameters of the shared layer can be made consistent by obtaining the parameters of the shared layer in the optimal global face attribute recognition model, which simplifies the amount of calculation and ensures the degree of fusion.

[0079] In another embodiment of the present application, in step S1, after the monitoring system performs image acquisition and obtains the face image, it further includes the following steps:

[0080] S1.1, divide the facial image into a training set and a test set in a ratio of 9:1, wherein the training set is used for image classification and preprocessing, and the test set is used to input the test results into the final facial attribute recognition model.

[0081] It is understandable that the test set may also be enhanced, such as image scaling, flipping and rotation, affine transformation, and adjustment of image brightness, so as to increase the data volume of the test set.

[0082] See also Figure 6 The present application also provides a terminal, which includes a facial attribute recognition model, and the terminal also includes a memory, a processor, and a recognition program stored in the memory and executable on the processor. The facial attribute recognition model is connected to the processor, and when the recognition program is executed by the processor, the steps of the facial attribute recognition method described above are implemented.

[0083] The terminal may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal may also include input and output devices, a network access device, a bus, etc.

[0084] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal, and various interfaces and lines are used to connect various parts of the entire terminal.

[0085] Compared with the prior art, the terminal provided in the present application can simultaneously identify multiple attributes of a face by adopting the above-mentioned face attribute recognition method, shorten the time for the model to recognize attributes, reduce the memory resources occupied, and at the same time improve the recognition rate of various attributes of the face.

[0086] If the module / unit integrated in the terminal is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0087] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for recognizing facial attributes, characterized in that: include: The monitoring system collects images and obtains facial images; Classifying and preprocessing the face images according to global face attributes and multiple local face attributes to generate a global face attribute training data set and multiple local face attribute data sets; Designing an initial face attribute recognition model, the initial face attribute recognition model comprising a shared layer, a global face attribute branch, and a plurality of local face attribute branches, saving a global face attribute recognition model composed of the shared layer and the global face attribute branches, and saving a local face attribute recognition model composed of the shared layer and the local face attribute branches; Inputting the global face attribute training data set into the global face attribute recognition model for training to obtain an optimal global face attribute recognition model; inputting the plurality of local face attribute training data sets into the local face attribute recognition models respectively, and loading the parameters of the optimal global face attribute recognition model for training to obtain a plurality of optimal local face attribute recognition models; The optimal global face attribute recognition model and the multiple optimal local face attribute recognition models are merged to obtain a final face attribute recognition model; Input the test set into the final face attribute recognition model to obtain the corresponding face attribute recognition results.

2. The method for face attribute recognition according to claim 1, characterized in that: The global face attributes include gender attributes.

3. The method for face attribute recognition according to claim 2, characterized in that: The local facial attributes include at least one of glasses, a mask and a hat.

4. The method for face attribute recognition according to claim 1, characterized in that: The pre-processing steps include: Performing enhancement processing on the facial image; The enhanced face image is converted into a model input size.

5. The method for facial attribute recognition according to claim 1, characterized in that: The initial face attribute recognition model uses the EfficientNet model as a benchmark model, the first N layers of the initial face attribute recognition model are the shared layers, the Nth layer of the initial face attribute recognition model is a node of a multi-branch model, and the Nth layer is followed by a global face attribute branch and a variety of local face attribute branches.

6. The method for facial attribute recognition according to claim 1, characterized in that: The step of inputting the global face attribute training data set into the global face attribute recognition model for training comprises: Set network parameters; By continuously reducing the cross entropy loss until the model converges; The loss function of the network is: Among them, m represents the number of samples, n represents the number of categories, P is a function with a value of 1 or 0, which means that if the true category of sample i is equal to category j, it takes 1, otherwise it takes 0, and q represents the predicted probability that sample i belongs to category j, and its value range is 0 to 1.

7. The method for facial attribute recognition according to claim 6, characterized in that: The step of loading the parameters of the optimal global face attribute recognition model comprises: Obtaining parameters of a shared layer in the optimal global face attribute recognition model; Freezing parameters of a shared layer in the local face attribute recognition model; The network parameters of the local face attribute branch are set for training.

8. The method for facial attribute recognition according to claim 1, characterized in that: After the monitoring system performs image acquisition and obtains the face image, it also includes the following steps: The facial images are divided into a training set and a test set in a ratio of 9:1, wherein the training set is used for image classification and preprocessing, and the test set is used to input the test results into the final facial attribute recognition model.

9. A terminal, characterized in that: The terminal includes a face attribute recognition model, a memory, a processor, and a recognition program stored in the memory and executable on the processor. The face attribute recognition model is connected to the processor, and when the recognition program is executed by the processor, the steps of the face attribute recognition method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the steps of the face attribute recognition method according to any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Face attribute recognition system and method based on multiple areas of face

    CN110069994A

  • Image processing method and device, electronic equipment and storage medium

    CN111144348A

  • Face attribute recognition method and device, storage medium and intelligent equipment

    CN113536845A

  • Feature extraction model training method, facial recognition method, apparatus, device and medium

    WO2021174880A1