Face attribute recognition method and system based on self-distillation transformer

Abstract

The present application relates to a kind of face attribute recognition method and system based on self-distillation Transform, the method includes the following steps: A.to the face attribute data picture of acquisition is preprocessed;B.the data after pre-processing is input into Transform, utilize Transform to promote the information interaction between different levels features, to avoid important information in the process of network iteration is ignored;C.Transform is grouped, and the classification Token in the Tokens extracted by the encoder of each group Transform is used for self-distillation module;D.the classification Token that the last encoder group output in Transform is normalized after input to fully connected layer for face attribute recognition;E.the self-distillation module and face attribute recognition module are jointly trained;F.the network model obtained by training is used for face attribute recognition.The method and system enhance the information interaction between different encoder blocks, to avoid forgetting the information in the process of iteration in the encoder block, while improving the performance of face attribute recognition.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and particularly relates to a face attribute recognition method and system based on self-distillation Transformer. BACKGROUND

[0002] Face attribute recognition (FAR) methods based on convolutional neural networks have been widely applied in computer vision (CV) and pattern recognition, including image retrieval, face verification and face recognition. The task of face attribute recognition is to predict whether a facial image has multiple facial attributes, such as wearing a hat, wearing glasses and smiling. Inspired by the success of Transformer in natural language processing and machine translation tasks, many studies have recently attempted to apply Transformer to computer vision tasks and have achieved vigorous development. However, few studies have attempted to use encoders for facial attribute recognition. Therefore, the present application will attempt to apply Transformer to face attribute recognition and construct a new method and system for dealing with face attribute recognition.

[0003] In recent years, self-distillation as a way of knowledge distillation has been widely studied. Although self-distillation is a distillation method of knowledge distillation, unlike general knowledge distillation, self-distillation does not require additional teacher network pre-training and is not used for model compression. Self-distillation can be used for model precision improvement and knowledge transfer between different modules within the model.

[0004] Visual Transformer divides an image into blocks, which are then input into an encoder to extract features for classification. Due to its simplicity and scalability, it has become a milestone in the application of Transformer in computer vision, and many works are based on traditional Transformer for improvement. However, they ignore the exchange of information between Transformer blocks and the interaction of information in different dimensions. SUMMARY

[0005] The purpose of the present application is to provide a face attribute recognition method and system based on self-distillation Transformer, which enhances the interaction of information between different encoder blocks to avoid forgetting the information in the encoder blocks during the iteration process, while improving the performance of face attribute recognition.

[0006] To achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows: a face attribute recognition method based on self-distillation Transformer, comprising the following steps:

[0007] A. Preprocessing the acquired face attribute data pictures;

[0008] B. The preprocessed data is input into the Transformer, and the Transformer is used to promote the information interaction between different levels of features to avoid ignoring important information in the process of network iteration;

[0009] C. The Transformers are grouped, and the classification Tokens in the Tokens extracted by the encoder of each group of Transformers are used in the self-distillation module;

[0010] D. The classification Tokens output by the last encoder group in the Transformer are normalized and input into the fully connected layer for face attribute recognition;

[0011] E. The self-distillation module and the face attribute recognition module are jointly trained;

[0012] F. The trained network model is used for face attribute recognition.

[0013] Further, the step A specifically comprises the following steps:

[0014] A1. Obtain face pictures and face attribute recognition labels respectively;

[0015] A2. For each face image to be input divide it into a series of flattened image blocks where N represents the number of image blocks, P is the size of the block, and C represents the channel number of the image block; then, through a trainable linear projection map the image blocks to a D-dimensional embedding space; concatenate the classification Tokens with the Tokens mapped after the image blocks are flattened; the position information corresponding to each Token is embedded into all Tokens; after preprocessing, the embedding information of the image is represented as:

[0016]

[0017] wherein, represents the jth flattened image block.

[0018] Further, the specific implementation method of step B is:

[0019] In order to avoid the encoder ignoring the information in the shallow layer in the iteration process, in each group of encoders of the Transformer, the information key and value of the multi-head self-attention of the previous encoder are input into the time zone self-attention of each layer of the encoder for information interaction in different encoder blocks; assuming that the Transformer is composed of G encoder groups, each encoder group contains a stack of L encoders, where L denotes the total number of encoders in the Transformer; each encoder consists of a local self-attention and a multi-layer perceptron; the output of the l-th layer is denoted as:

[0020] z′ l = ZSA(LN(z l-1 ), K l-1 , V l-1 ) + z l-1 , l = 1,..., L

[0021] z l = MLP(LN(z′ l )) + z′ l , l = 1,..., L

[0022] where LN(·) denotes the layer normalization operation; z′ l and z l are the hidden features in the encoder; K l-1 and V l-1 are the obtained keys and values from the (l-1)-th encoder; the local self-attention in the l-th encoder also receives the query Q l , the keys K l and the values V l , and the local self-attention in the l-th layer of the encoder contains the keys K l-1 and the values V l-1 of the (l-1)-th layer to increase the interaction of information of different layers, so that the local self-attention can learn the relationship between the feature information of different time zones; the local self-attention in the l-th encoder is denoted as:

[0023] ZSA(LN(z l-1 ), K l-1 , V l-1 )

[0024] = Concat(head0,...,head N )W O

[0025] where is a learnable linear mapper, and Concat(·) is a concatenation operation; and head i is denoted as:

[0026]

[0027] where and are linear mappers of head i ; the self-attention operation of Attn(·) is calculated according to the following formula:

[0028]

[0029] wherein

[0030] Further, the specific implementation method of the step C is:

[0031] A self-distillation mechanism based on classification Token is adopted to promote the interaction of important information between different encoder groups; the self-distillation mechanism based on classification Token distills the information of the classification Token obtained from the last encoder group into the classification Token of other shallow encoder groups; the total L encoders in the Transformer are divided into G groups, each group has encoders, and the output of the encoder of the jth group is Let wherein represents the 0th Token of ; the classification Token obtained by using the last group of encoders, i.e. is used as a teacher to affect the classification Token generated from the gth group, i.e. wherein g={1,...,G-1}; then the self-distillation loss function based on Kullback-Leibler divergence KL(·) is described as follows:

[0032]

[0033] wherein τ is a temperature parameter, which is set to G=4 and τ=2.

[0034] Further, the specific implementation method of the step D is:

[0035] The classification Token of the last encoder group of the Transformer, i.e. is input into the layer normalization LN(·) and the fully connected layer FC(·) to obtain the predicted value of the facial attribute Given the predicted value of the facial attribute and the real value y, the loss calculation of the facial attribute recognition is as follows:

[0036]

[0037] wherein, M is the number of training images, and A is the number of facial attributes.

[0038] Further, the specific implementation method of the step E is:

[0039] The self-distillation module and the facial attribute recognition module are jointly trained, which is represented as follows:

[0040] Ltotal = L FAC + lambda * L self

[0041] where lambda is the self-distillation loss L self of the hyperparameters.

[0042] The application further provides a face attribute recognition system based on a self-distillation Transformer, comprising a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor, when the processor executes the computer program instructions, the above-mentioned method steps can be realized.

[0043] Compared with the prior art, the application has the following beneficial effects: the application proposes a face attribute recognition method and system based on a self-distillation Transformer, first proposes a new face attribute recognition system based on a Transformer, realizes the interaction between different Transformer encoders, and avoids ignoring the effective information between the encoder groups of the Transformer in the iteration process. In addition, the application further proposes a new self-distillation mechanism based on classification Token, which interacts with the important information between different encoders through attention, and distills the information of the classification Token obtained from the last encoder group of the Transformer to other relatively shallow encoder groups. Therefore, the face attribute recognition method and system proposed by the application have more excellent face attribute recognition performance. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a network model framework diagram of an embodiment of the application.

[0045] Figure 2 is a framework diagram of an encoder group of the Transformer in the embodiment of the application. DETAILED DESCRIPTION

[0046] The application will be further described below in combination with the drawings and embodiments.

[0047] It should be pointed out that the following detailed description is exemplary and is intended to provide further description of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the present application belongs.

[0048] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0049] As shown in Figure 1 , 2 The embodiment provides a face attribute recognition method based on self-distillation Transformer, and comprises the following steps:

[0050] A. Preprocessing the obtained face attribute data picture.

[0051] In the embodiment, step A specifically comprises the following steps:

[0052] A1. Obtaining a face picture and a face attribute recognition label respectively.

[0053] A2. For each face image to be input It is divided into a series of flattened image blocks Where N represents the number of image blocks, P is the size of the block, and C represents the channel number of the image block. Then, the image block is mapped into a D-dimensional embedding space through a trainable linear projection The classification Token is a learnable vector, which is spliced with the Tokens mapped after the image block is flattened. The classification Token is used for self-distillation of the encoder group of the Transformer; the position information corresponding to each Token is embedded into all Tokens; after preprocessing, the embedding information of the image is represented as:

[0054]

[0055] Wherein, represents the jth flattened image block.

[0056] B. Inputting the preprocessed data into the Transformer, and using the Transformer to promote the information interaction between different levels of features to avoid ignoring important information in the process of network iteration.

[0057] In the embodiment, the specific implementation method of step B is:

[0058] To avoid the encoder ignoring the information in the shallow layers during the iteration process, in each group of encoders of the Transformer, the encoder of each layer will input the information keys and values of the multi-head self-attention of the previous encoder into the time zone self-attention (ZSA) for information interaction in different encoder blocks. Assuming that the Transformer is composed of G encoder groups, each encoder group contains stacked encoders, where L represents the total number of encoders in the Transformer. Each encoder is composed of a time zone self-attention (ZSA) and a multi-layer perceptron (MLP). Therefore, the output of the lth layer can be represented as follows:

[0059] z′ l = ZSA(LN(z l-1 ), K l-1 , V l-1 ) + z l-1 , l = 1,..., L

[0060] z l = MLP(LN(z′ l )) + z′ l , l = 1,..., L

[0061] where LN(·) represents the layer normalization operation; z′ l and z l are the hidden features in the encoder; K l-1 and V l-1 are the keys and values obtained by the (l-1)th encoder, respectively. The time zone self-attention (ZSA) is similar to the general multi-head self-attention, except that the time zone self-attention in the lth encoder also receives the query Q l , the key K l , and the value V l . The difference is that the time zone self-attention in the lth layer of the encoder contains the keys K l-1 and values V l-1 of the (l-1)th layer. Such a design increases the interaction of information between different layers, so that the time zone self-attention can learn the relationship between the feature information of different time zones. The time zone self-attention in the lth encoder can be represented as:

[0062] ZSA(LN(z l-1 ), K l-1 , V l-1 )

[0063] = Concat(head0,...,head N )W O

[0064] where is a learnable linear projector, Concat(·) is a concatenation operation. And, head i can be represented as:

[0065]

[0066] where and is a linear projector of head i The self-attention operation of Attn(·) can be calculated according to the following formula:

[0067]

[0068] where

[0069] C. Grouping the Transformers, and using the classification tokens of the tokens extracted by the encoders of each group of Transformers in a self-distillation module.

[0070] In the embodiment, the specific implementation method of the step C is:

[0071] In order to interact important information between different encoder groups, a new self-distillation mechanism based on classification tokens is adopted. The mechanism distills the information of the classification tokens obtained from the last encoder group into the classification tokens of other shallow encoder groups. Specifically, the total L encoders in the Transformer are divided into G groups, and each group has encoders, so the output of the jth group of encoders is We let where represents the 0th token of We use the classification token obtained by the last group of encoders, that is as a teacher to affect the classification token generated from the gth group, that is where g = {1,..., G-1}. Therefore, the self-distillation loss function based on Kullback-Leibler divergence (KL(·)) can be described as follows:

[0072]

[0073] where τ is a temperature parameter, which is set to G = 4 and τ = 2.

[0074] D. Normalizing the classification tokens output by the last encoder group in the Transformer and inputting them to a fully connected layer for face attribute recognition.

[0075] In the embodiment, the step D is specifically implemented as follows:

[0076] The classification Token of the last encoder group of the Transformer is input into the layer normalization LN(·) and the fully connected layer FC(·) to obtain the predicted value of the facial attribute Therefore, given the predicted value of the facial attribute and the real value y, the loss of the facial attribute recognition can be calculated as follows:

[0077]

[0078] wherein, M is the number of training images, and A is the number of facial attributes.

[0079] E. Joint training of the self-distillation module and the facial attribute recognition module, which is represented as follows:

[0080] L total =L FAC +λ·L self

[0081] wherein λ is a hyperparameter of the self-distillation loss L self .

[0082] F. Facial attribute recognition by using the network model obtained by training.

[0083] The application further provides a facial attribute recognition system based on self-distillation Transformer, which comprises a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor, and when the processor executes the computer program instructions, the method steps as described above can be realized.

[0084] In the embodiment, the facial attribute recognition results of the method proposed in the application and other methods are compared. Table 1 is a comparison of the results of the method proposed in the application and other facial attribute recognition methods on the CelebA dataset. Table 2 is a comparison of the results of the method proposed in the application and other facial attribute recognition methods on the LFWA dataset.

[0085] Table 1 comparison of the recognition results of the method proposed in the application and other methods on the CelebA dataset

[0086] Attributes (% accuracy) PANDA LNets+ANet MCNN-AUX AFFAIR MCFA SPLITFACE DMM-CNN The present method Moustache 88.00 91.00 94.51 94.79 94.00 93.13 94.84 94.94 Cupid's bow 78.00 79.00 83.42 83.99 83.00 82.56 84.57 83.56 Attractiveness 81.00 81.00 83.06 83.21 83.00 82.76 83.37 83.66 Under-eye bags 79.00 79.00 84.92 85.16 85.00 84.86 85.81 85.49 Bald 96.00 98.00 98.90 98.93 99.00 98.03 99.03 99.11 Bangs 92.00 95.00 96.05 96.15 96.00 95.71 96.22 96.16 Big lips 67.00 68.00 71.47 71.84 72.00 69.28 72.93 72.20 Big nose 75.00 78.00 84.53 84.54 84.00 83.81 84.78 85.42 Black hair 85.00 88.00 89.78 90.11 89.00 89.03 90.50 90.72 Blonde hair 93.00 95.00 96.01 96.14 96.00 95.76 96.13 96.31 Bloodshot eyes 86.00 84.00 96.17 96.43 96.00 95.96 96.40 96.39 Brown hair 77.00 80.00 89.15 89.55 88.00 88.25 89.46 89.68 Bushy eyebrows 86.00 90.00 92.84 92.89 92.00 92.66 93.01 93.06 Full 86.00 91.00 95.67 95.78 96.00 95.94 95.86 96.05 Double chin 88.00 92.00 96.32 96.41 96.00 95.80 96.39 96.49 Eyes 98.00 99.00 99.63 99.71 100.00 99.51 99.69 99.72 Goatee 93.00 95.00 97.24 97.60 97.00 96.68 97.63 97.38 Grey hair 94.00 97.00 98.20 98.24 98.00 97.45 98.27 98.33 Heavy makeup 90.00 90.00 91.55 91.92 92.00 91.59 91.85 92.03 High cheekbones 86.00 88.00 87.58 88.13 87.00 87.61 87.73 87.86 Male 97.00 98.00 98.17 98.46 98.00 97.95 98.29 98.82 Open mouth 93.00 92.00 93.74 94.27 93.00 93.78 94.16 94.20 Moustache (upper lip) 93.00 95.00 96.88 97.03 97.00 95.86 97.03 96.87 Small eyes 84.00 81.00 87.23 87.84 87.00 86.88 87.73 87.73 Moustache (chin) 93.00 95.00 96.05 96.37 96.00 96.17 96.41 96.53 Oval face 65.00 66.00 75.84 76.31 75.00 74.93 75.89 76.12 Pale skin 91.00 91.00 97.05 97.18 97.00 97.00 97.00 97.06 Pointed nose 71.00 72.00 77.47 77.49 77.00 76.47 77.19 78.18 Receding hairline 85.00 89.00 93.81 93.75 94.00 92.25 94.12 94.03 Rosy cheeks 87.00 90.00 95.16 95.32 95.00 94.79 95.32 95.21 Sideburns 93.00 96.00 97.85 97.93 98.00 97.17 97.91 97.86 Smiling 92.00 92.00 92.73 93.23 93.00 92.70 93.22 93.17 Straight hair 69.00 73.00 83.58 83.46 85.00 80.41 84.72 84.73 Wavy hair 77.00 80.00 83.91 84.01 85.00 81.70 86.01 85.52 Wearing earrings 78.00 82.00 90.43 90.51 90.00 89.44 90.78 90.96 Wearing hats 96.00 99.00 99.05 99.12 99.00 98.74 99.12 99.11 Wearing lipstick 93.00 93.00 94.11 94.04 94.00 93.21 94.49 94.29 Wearing necklaces 67.00 71.00 86.63 86.48 88.00 85.61 88.03 86.16 Wearing ties 91.00 93.00 96.51 95.25 97.00 96.05 97.15 95.79 Young 84.00 87.00 88.48 88.44 88.00 88.01 88.98 89.66 Mean accuracy 85.43 87.33 91.29 91.45 91.23 90.61 91.70 91.66 Parameter amount - ＞100M 16M - 260M 26.09M 90.77M 85.83M

[0087] Table 2 comparison of the recognition results of the method proposed in the application and other methods on the LFWA dataset

[0088] Attributes (% accuracy) PANDA LNets+ANet MCNN-AUX AFFAIR MCFA SPLITFACE DMM-CNN The present method Moustache 84.00 84.00 77.06 77.98 75.00 77.59 79.18 78.47 Cupid's bow 79.00 82.00 81.78 81.63 79.00 81.72 82.70 81.36 Attractiveness 81.00 83.00 80.31 80.31 77.00 80.16 81.10 81.08 Under-eye bags 80.00 83.00 83.48 83.12 79.00 82.62 82.70 83.59 Bald 84.00 88.00 91.94 91.67 91.00 91.88 91.96 91.89 Bangs 84.00 88.00 90.08 91.08 89.00 90.71 91.30 91.10 Big lips 73.00 75.00 79.24 78.91 75.00 78.97 79.82 79.87 Big nose 79.00 81.00 84.98 84.20 81.00 83.13 83.67 83.53 Black hair 87.00 90.00 92.63 91.66 91.00 92.49 91.55 91.49 Blonde hair 94.00 97.00 97.41 97.56 97.00 97.47 97.17 97.46 Bloodshot eyes 74.00 74.00 85.23 86.82 86.00 86.42 87.58 87.79 Brown hair 74.00 77.00 80.85 79.35 77.00 80.93 81.56 80.70 Bushy eyebrows 79.00 82.00 84.97 84.90 76.00 84.26 85.33 85.21 Full 69.00 73.00 76.86 75.71 74.00 76.06 77.66 78.02 Double chin 75.00 78.00 81.52 81.00 77.00 80.49 80.98 82.16 Eyes 89.00 95.00 91.30 92.25 91.00 91.50 92.83 92.68 Goatee 75.00 78.00 82.97 83.56 80.00 83.01 82.82 84.45 Grey hair 81.00 84.00 88.93 88.14 88.00 88.46 89.38 89.93 Heavy makeup 93.00 95.00 95.85 95.86 94.00 95.39 95.68 95.53 High cheekbones 86.00 88.00 88.38 88.85 85.00 88.34 88.13 88.30 Male 92.00 94.00 94.02 93.94 93.00 92.60 94.14 94.02 Open mouth 78.00 82.00 83.51 81.48 78.00 82.50 84.45 82.44 Moustache (upper lip) 87.00 92.00 93.43 93.71 91.00 92.97 94.46 94.03 Small eyes 73.00 81.00 82.86 82.08 78.00 82.75 83.67 83.58 Moustache (chin) 75.00 79.00 82.15 81.34 79.00 80.77 82.48 82.53 Oval face 72.00 74.00 77.39 79.32 74.00 76.80 76.94 78.02 Pale skin 84.00 84.00 93.32 91.24 82.00 90.97 91.86 92.51 Pointed nose 76.00 80.00 84.14 84.94 80.00 84.20 84.51 84.39 Receding hairline 84.00 85.00 86.25 86.13 85.00 84.90 86.30 86.96 Rosy cheeks 73.00 78.00 87.92 87.56 85.00 87.08 86.44 88.31 Sideburns 76.00 77.00 83.13 83.10 78.00 81.76 82.99 83.02 Smiling 89.00 91.00 91.83 91.51 88.00 90.80 92.24 91.92 Straight hair 73.00 76.00 78.53 78.58 77.00 78.91 79.20 80.05 Wavy hair 75.00 76.00 81.61 79.72 79.00 78.28 79.87 81.08 Wearing earrings 92.00 94.00 94.95 95.07 93.00 94.75 94.14 94.76 Wearing hats 82.00 88.00 90.07 89.84 91.00 90.23 90.84 90.96 Wearing lipstick 93.00 95.00 95.04 94.77 94.00 94.07 95.11 94.89 Wearing necklaces 86.00 88.00 89.94 89.77 89.00 89.59 89.47 90.52 Wearing ties 79.00 79.00 80.66 80.71 82.00 81.40 81.28 81.83 Young 82.00 86.00 85.84 85.93 87.00 85.68 88.94 88.88 Mean accuracy 81.03 83.03 86.31 86.13 83.63 85.82 86.56 86.73 Parameter amount - ＞100M 16M - 260M 26.09M 90.77M 85.83M

[0089] wherein:​

[0090] PANDA corresponds to the method proposed by N. Zhang et al. (N. Zhang, M. Paluri, M. Ranzato, T. Darrell, L. Bourdev, Panda: Pose aligned networks for deep attribute modeling, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 1637-1644.);

[0091] LNets+ANet corresponds to the method proposed by Z. Liu et al. (Z. Liu, P. Lno, X. Wang, X. Tang, Deep learning face attributes in the wild, in: Proceedings of the IEEE international conference on computer vision, 2015, pp. 3730-3738.);

[0092] MCNN-AUX corresponds to the method proposed by E. M. Hand et al. (E. M. Hand, R. Chellappa, Attributes for improved attributes: A multitask network utilizing implicit and explicit relationships for facial attribute classification, in: Proceedings of the 31st (AAAI) Conference on Artificial Intelligence, 2017, pp. 4068-4074.);

[0093] AFFAIR corresponds to the method proposed by J. Li et al. (J. Li, F. Zhao, J. Feng, S. Roy, S. Yan, T. Sim, Landmark free face attribute prediction, IEEE Transactions on Image Processing 27(9) (2018) 4651-4662.);

[0094] MCFA corresponds to the method proposed by N. Zhuang et al. (N. Zhuang, Y. Yan, S. Chen, H. Wang, Multi-task learning of cascaded cnn for facial attribute classification, in: 2018 24th International Conference on Pattern Recognition (ICPR), IEEE, 2018, pp. 2069-2074.);

[0095] SPLITFACE corresponds to the method proposed by U. Mahbub et al. (U. Mahbub, S. Sarkar, R. Chellappa, Segment-based methods for facial attribute detection from partial faces, IEEE Transactions on Affective Computing 11(4) (2018) 601-613.);

[0096] DMM-CNN corresponds to the method proposed by L. Mao et al. (L. Mao, Y. Yan, J.-H. Xue, H. Wang, Deep multi-task multi-label cnn for effective facial attribute classification, IEEE Transactions on Affective Computing).

[0097] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems, or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) having computer-usable program code embodied in the medium.

[0098] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0099] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0100] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0101] The above descriptions are only preferred embodiments of the present application, and are not intended to limit the present application to other forms. Any person skilled in the art can make modifications or improvements on the basis of the above disclosed technical content without departing from the technical scope of the present application. Any simple modification, equivalent change and improvement made on the basis of the above embodiments without departing from the technical scope of the present application shall fall within the scope of protection of the present application.

Claims

1. A face attribute recognition method based on self-distillation Transformer, characterized in that, The method comprises the following steps: A. Preprocessing the acquired face attribute data picture; B. Inputting the preprocessed data into the Transformer, and promoting information interaction between different levels of features by using the Transformer to avoid ignoring important information in the process of network iteration; C. Grouping the Transformers, and using the classification Token in the Tokens extracted by the encoder of each group of Transformers for a self-distillation module; D. Normalizing the classification Token output by the last encoder group in the Transformer and inputting it into a fully connected layer for face attribute recognition; E. Jointly training the self-distillation module and the face attribute recognition module; F. Using the trained network model to perform face attribute recognition. The specific implementation method of step B is: To avoid the encoder ignoring the information in the shallow layer in the iteration process, in each group of encoders of the Transformer, the information keys and value common inputs of the multi-head self-attention of the previous encoder are input into the time zone self-attention of each layer of the encoder for information interaction in different encoder blocks; assuming that the Transformer is composed of encoder groups, each encoder group contains stacked encoders, wherein denotes the total number of encoders in the Transformer; each encoder is composed of a time zone self-attention and a multi-layer perception; the output of the layer is represented as: in, This indicates a hierarchical normalization operation. and These are hidden features in the encoder; and They are the first The keys and values ​​obtained by the encoder; the time zone self-attention in the first... The time zone self-attention in each encoder also receives queries. ,key Sum The time zone self-attention in the encoder of layer l contains the first... Layer Key Sum To increase the interaction of information at different levels, enabling time zone self-attention to learn the relationships between feature information from different time zones; The time zone self-attention in each encoder is represented as: wherein is a learnable linear mapper, is a concatenation operation; and, is represented as: wherein , and are linear mappers; The self-attention operation of is computed according to the following formula: wherein , .

2. The self-distilled Transformer-based face attribute recognition method of claim 1, wherein, The step A specifically comprises the following steps: A1. Acquire face pictures and face attribute recognition labels respectively; A2. For each face image to be input It is divided into a series of flattened image blocks. ,in Indicates the number of image patches. It is the size of the block. This represents the number of channels in an image patch; then, it is processed through a trainable linear projection. Image blocks Mapped to In the dimensional embedding space; the classification tokens are concatenated with the tokens mapped after the image patches are flattened; the location information corresponding to each token. Embedded into all tokens; after preprocessing, the embedding information of the image is represented as: wherein represent the flattened image blocks. represent the flattened image blocks.

3. The self-distillation Transformer-based face attribute recognition method of claim 1, wherein, The specific implementation method of step C is: A self-distillation mechanism based on classification tokens is adopted to facilitate the interaction of important information between different encoder groups; the self-distillation mechanism based on classification tokens distills the information of the classification tokens obtained from the last encoder group into the classification tokens of other shallow encoder groups; a total of encoders in the Transformer are divided into groups, each group has encoders, and the output of the encoder in the first group is ; let , where represents the 0th Token of ; the classification tokens obtained by using the last group of encoders, i.e. , are used as teachers to affect the classification tokens generated from the first group, i.e. , where ; then the self-distillation loss function based on Kullback-Leibler divergence is described as follows: wherein is a temperature parameter, set to and .

4. The self-distillation Transformer-based face attribute recognition method of claim 1, wherein, The specific implementation method of step D is: The classification Token of the last encoder group of the Transformer, i.e. , is input to a layer normalization and a fully connected layer to obtain the predicted value of the facial attribute ; Predicted values for given facial attributes and true values The loss calculation for facial attribute recognition is as follows: wherein, ; is the number of training images, is the number of face attributes.

5. The self-distillation Transformer-based face attribute recognition method of claim 4, wherein, The specific implementation method of step E is: Jointly training the self-distillation module and the face attribute recognition module is represented as follows: wherein is a hyperparameter of the self-distillation loss is a hyperparameter of the self-distillation loss 6. A face attribute recognition system based on self-distillation Transformer, characterized in that, The computer program instructions stored in the memory and capable of being executed by the processor can implement the method steps of any one of claims 1-5 when the processor executes the computer program instructions.

Images