Face image recognition method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a face image recognition method and device, electronic equipment and a storage medium. The face image recognition method comprises the steps that a to-be-recognized face image is acquired; mapping the face image to be recognized to generate a face image token vector sequence; performing cross attention processing on the face image token vector sequence and a preset attribute token feature vector sequence to generate a face feature sequence; converting the face feature sequence into recognition features; and processing the face image to be recognized based on the recognition features. According to the embodiment of the invention, the method can effectively improve the recognition capability for the image when the age span is large, the image is blurred, and the face angle is large.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a face image recognition method, a face image recognition device, an electronic device and a storage medium. Background Art

[0002] At present, in the relevant technologies, large age spans, blurred images, and large angles of faces pose serious challenges to face recognition systems. When faced with such scenarios, the recall rate is often low, resulting in recognition system failure. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a face image recognition method, a face image recognition device, an electronic device and a storage medium that overcome the above problems or at least partially solve the above problems.

[0004] In order to solve the above problems, in a first aspect of the present invention, an embodiment of the present invention discloses a face image recognition method, comprising:

[0005] Obtain a face image to be recognized;

[0006] Mapping the face image to be recognized to generate a face image token vector sequence;

[0007] Performing cross-attention processing on the face image token vector sequence and the preset attribute token feature vector sequence to generate a face feature sequence;

[0008] Converting the facial feature sequence into recognition features;

[0009] The face image to be identified is processed based on the identification features.

[0010] Optionally, the step of mapping the face image to be identified to generate a face image token vector sequence includes:

[0011] Segmenting the face image to be recognized to generate multiple image blocks;

[0012] Mapping the image block to generate a face image token vector;

[0013] The face image token vectors are combined to generate a face image token vector sequence.

[0014] Optionally, the preset attribute token feature vector sequence includes attribute token feature vectors corresponding to multiple different facial attributes, and the step of performing cross-attention processing on the facial image token vector sequence and the preset attribute token feature vector sequence to generate a facial feature sequence includes:

[0015] Interacting a face image token vector in a face image token vector sequence with an attribute token feature vector in a preset attribute token feature vector sequence to generate a plurality of face features;

[0016] The facial features are combined to generate a facial feature sequence.

[0017] Optionally, the step of performing cross-attention processing on the facial image token vector sequence and the preset attribute token feature vector sequence to generate a facial feature sequence further includes:

[0018] Position encoding the face image token vector and the attribute token feature vector;

[0019] The encoded face image token vector and the encoded attribute token feature vector are used to perform the step of interacting the face image token vector in the face image token vector sequence with the attribute token feature vector in the preset attribute token feature vector sequence to generate multiple face features.

[0020] Optionally, the attribute token feature vector is determined by performing knowledge distillation on labeled facial attribute information.

[0021] Optionally, the step of converting the facial feature sequence into recognition features includes:

[0022] The facial features are fused to obtain identification features.

[0023] Optionally, the method further comprises:

[0024] Get the training image set and initial model;

[0025] The initial model is trained using the training image set to determine a face recognition model; the face recognition model is used to execute the step of mapping the face image to be recognized to generate a face image token vector sequence, and / or execute the step of cross-attention processing the face image token vector sequence and a preset attribute token feature vector sequence to generate a face feature sequence, and / or execute the step of converting the face feature sequence into recognition features.

[0026] In a third aspect of the present invention, an embodiment of the present invention discloses a facial image recognition device, comprising:

[0027] An acquisition module, used to acquire a face image to be recognized;

[0028] A first processing module, used for mapping the face image to be identified to generate a face image token vector sequence;

[0029] A second processing module is used to perform cross-attention processing on the face image token vector sequence and the preset attribute token feature vector sequence to generate a face feature sequence;

[0030] A third processing module, used for converting the facial feature sequence into recognition features;

[0031] The recognition module is used to process the face image to be recognized based on the recognition feature.

[0032] According to a third aspect of the present invention, an embodiment of the present invention discloses an electronic device, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the facial image recognition method as described above when executed by the processor.

[0033] In a fourth aspect of the present invention, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the facial image recognition method described above are implemented.

[0034] The embodiments of the present invention include the following advantages:

[0035] The embodiment of the present invention obtains a face image to be identified; maps the face image to be identified to generate a face image token vector sequence; performs cross-attention processing on the face image token vector sequence and a preset attribute token feature vector sequence to generate a face feature sequence; converts the face feature sequence into identification features; and processes the face image to be identified based on the identification features. By making full use of the attribute information corresponding to the face image data, the face image token vector sequence and the preset attribute token feature vector sequence are cross-attention processed. The attributes in the preset attribute token feature vector sequence can correspond to age, blur, and angle, so that they are robust to these ages, blurs, and angles, and the obtained face feature sequence is associated with these attributes. Therefore, when faced with problems such as large age spans, blurred images, and large face angles, face images can still be correctly identified, thereby improving the accuracy of face image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flowchart of a facial image recognition method embodiment of the present invention;

[0037] Figure 2 is a flowchart of another embodiment of a facial image recognition method of the present invention;

[0038] Figure 3 is a schematic diagram of a transformer architecture of an example of a face image recognition method of the present invention;

[0039] Figure 4 It is a block diagram of an implementation architecture of an example of a face image recognition method of the present invention;

[0040] Figure 5 It is a structural block diagram of an embodiment of a face image recognition device of the present invention;

[0041] Figure 6 is a structural block diagram of an electronic device provided by an embodiment of the present invention;

[0042] Figure 7 It is a structural block diagram of a storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Reference Figure 1 , shows a flowchart of a facial image recognition method embodiment of the present invention, the facial image recognition method may specifically include the following steps:

[0045] Step 101, obtaining a face image to be recognized;

[0046] The face image to be identified can be obtained.

[0047] Step 102, mapping the face image to be recognized to generate a face image token vector sequence;

[0048] Each part of the face image to be recognized can be mapped to generate a face image token vector sequence.

[0049] Step 103, cross-attention processing is performed on the facial image token vector sequence and the preset attribute token feature vector sequence to generate a facial feature sequence;

[0050] Then, the information in the face image token vector sequence and the preset attribute token feature vector sequence is cross-attention processed to obtain the face feature sequence.

[0051] Step 104, converting the facial feature sequence into recognition features;

[0052] Then convert the facial feature sequence into the recognition features of the face in the image.

[0053] Step 105: Process the face image to be identified based on the identification features.

[0054] After obtaining the recognition features, the face image to be identified is processed accordingly based on the recognition features to obtain the required image data, semantic information, tracking target, etc.

[0055] The embodiment of the present invention obtains a face image to be identified; maps the face image to be identified to generate a face image token vector sequence; performs cross-attention processing on the face image token vector sequence and a preset attribute token feature vector sequence to generate a face feature sequence; converts the face feature sequence into identification features; and processes the face image to be identified based on the identification features. By making full use of the attribute information corresponding to the face image data, the face image token vector sequence and the preset attribute token feature vector sequence are cross-attention processed. The attributes in the preset attribute token feature vector sequence can correspond to age, blur, and angle, so that they are robust to these ages, blurs, and angles, and the obtained face feature sequence is associated with these attributes. Therefore, when faced with problems such as large age spans, blurred images, and large face angles, face images can still be correctly identified, thereby improving the accuracy of face image recognition.

[0056] Reference Figure 2 , shows a flowchart of another embodiment of a face image recognition method of the present invention, wherein the face image recognition method may specifically include the following steps:

[0057] Step 201, obtaining a face image to be recognized;

[0058] When performing facial image recognition, a facial image to be recognized may be obtained first.

[0059] Step 202, segmenting the face image to be recognized to generate multiple image blocks;

[0060] The face image to be recognized can be segmented, and each segmented part is an image block, thereby generating multiple image blocks. The number of segments can be determined according to demand, and the embodiment of the present invention does not specifically limit this.

[0061] The input face image to be recognized is divided into blocks, and the corresponding block features are extracted. For example, the input face image to be recognized has a width of W, a height of H, and a number of channels of channel_dim. The image block has a height of P and a width of P. The total number of image blocks obtained at this time is N, that is, N = HW / P 2 .

[0062] Step 203, mapping the image block to generate a face image token vector;

[0063] The image blocks are then mapped, and the dimension mapping layer in the model can be used to map multiple image blocks into face image token vectors corresponding to multiple dimensions.

[0064] Step 204, combining the face image token vectors to generate a face image token vector sequence;

[0065] The combination of multiple face image token vectors is a face image token vector sequence.

[0066] Step 205, cross-attention processing is performed on the facial image token vector sequence and the preset attribute token feature vector sequence to generate a facial feature sequence;

[0067] The face image token vector sequence can be interacted with the information in the preset attribute token feature vector sequence so that the face image token vector is regressed based on different attributes to generate a face feature sequence.

[0068] Specifically, the preset attribute token feature vector sequence includes attribute token feature vectors corresponding to multiple different facial attributes, and the step of performing cross-attention processing on the facial image token vector sequence and the preset attribute token feature vector sequence to generate a facial feature sequence includes: interacting the facial image token vector in the facial image token vector sequence with the attribute token feature vector in the preset attribute token feature vector sequence to generate multiple facial features; and combining the facial features to generate a facial feature sequence. The attribute token feature vector is determined by performing knowledge distillation on the annotated facial attribute information.

[0069] The preset attribute token feature vector sequence includes attribute token feature vectors corresponding to multiple different facial attributes. Facial attributes include a variety of natural features related to the face, such as the position of facial features, whether glasses are worn, etc. The attribute token feature vector is determined by performing knowledge distillation on the annotated facial attribute information. For example, the attribute token feature vector can be M facial attribute token features with a dimension size of embed_dim. The embedded facial attribute token features are distilled using an existing attribute teacher model. The teacher model can be any CNN (convolutional neural network) network (such as VGG, ResNet, RegNet, etc.), or a network with a ViT architecture (such as Swin Transformer, etc.). The objective function used to supervise this module during the training phase is composed of a distillation loss function and a cross entropy loss function. The distillation loss can be soft label distillation or hard label distillation. Soft label distillation can refer to the following formula:

[0070] L distill_ =(1-α)L CE (ψ(Z s ),y)+ατ 2 KL(ψ(Z s / τ),ψ(Z t / τ))

[0071] Among them, Z t is the attribute feature vector output by the teacher model, Z s is the attribute feature vector output by the student model. τ represents the temperature parameter of distillation, and α represents the balanced divergence loss (Kullback–Leibler, KL) and the cross entropy loss (L CE ), and ψ is used to represent the Softmax function, where α takes a value in the range of [0,1].

[0072] Hard label distillation can refer to the following formula:

[0073]

[0074] Among them, y t is the category number predicted by the teacher model.

[0075] The face image token vector in the face image token vector sequence can be interacted with the attribute token feature vector in the preset attribute token feature vector sequence to generate multiple face features; the face feature sequence can be obtained by combining the obtained face features.

[0076] That is, the face image token vector in the face image token vector sequence can be matched and associated with the attribute token feature vector in the preset attribute token feature vector sequence, thereby establishing a correlation between the two different representations. As shown in the following formula:

[0077]

[0078] Among them, the query feature Q comes from the attribute features after distillation, while the key feature (Key, K) and value feature (Value, V) come from the face image block features output by the Transformer encoding module, and d represents the dimension size of the feature.

[0079] In Cross Attention, the query feature (Query, Q) comes from the attribute feature after distillation, while the key feature (Key, K) and value feature (Value, V) come from the face image block feature output by the Transformer encoding module. The output is the face attribute feature. The M face feature vectors found in the face feature are the face features.

[0080] In addition, the step of cross-attentionally processing the face image token vector sequence and the preset attribute token feature vector sequence to generate a face feature sequence also includes: position encoding the face image token vector and the attribute token feature vector; using the encoded face image token vector and the encoded attribute token feature vector to execute the step of interacting the face image token vector in the face image token vector sequence with the attribute token feature vector in the preset attribute token feature vector sequence to generate multiple face features.

[0081] In the embodiment of the present invention, the face image token vector and the attribute token feature vector can also be positionally encoded using a coding module of a transformer architecture. The transformer architecture can refer to Figure 3 , including input normalization layer (Norm), multi-head self-attention layer (Multi-HeadAttention, MHA), multi-layer perceptron (Multi-Layer Perceptron, MLP). Through encoding, the feature vectors corresponding to N image blocks and M embedded attribute feature vectors are obtained.

[0082] The encoded face image token vector and the encoded attribute token feature vector are then used to perform the step of interacting the face image token vector in the face image token vector sequence with the attribute token feature vector in the preset attribute token feature vector sequence to generate multiple face features, thereby obtaining the face features.

[0083] Step 206, converting the facial feature sequence into identification features;

[0084] The facial feature sequence can be converted into recognition features, so that the specific facial information can be determined.

[0085] Specifically, the step of converting the facial feature sequence into identification features includes: fusing the facial features to obtain identification features.

[0086] The obtained facial features can be fused to obtain identification features. The fusion method can be through MLP (multi-layer perceptron) fusion or two fully connected layers fusion.

[0087] Step 207: Process the facial image to be identified based on the identification features.

[0088] Finally, the recognition features can be used to process the face image to be recognized, completing the corresponding image processing.

[0089] Furthermore, a training image set and an initial model can be obtained; the initial model is trained using the training image set to determine a face recognition model; the face recognition model is used to execute the step of mapping the face image to be recognized to generate a face image token vector sequence, and / or, execute the step of cross-attention processing the face image token vector sequence and a preset attribute token feature vector sequence to generate a face feature sequence, and / or, execute the step of converting the face feature sequence into recognition features.

[0090] That is, a training image set can be obtained in the training stage, which contains label information (gt), wherein the label information includes face category information and face attribute information. The face category information is used to supervise face features, and the face attribute information is used to supervise embedded attribute token features. The initial model is trained with the training image set to obtain a face recognition model. Steps 202-206 are performed with the face recognition model, so that the image can be accurately recognized.

[0091] The embodiment of the present invention embeds attribute token features and allows the attributes to interact with the face in the image, so that the attribute information can be fully utilized, thereby effectively improving the recognition ability when the age span is large, the picture is blurred, and the face is at a large angle.

[0092] In order to make the implementation process of the embodiments of the present invention clear to those skilled in the art, some examples are used to illustrate the following:

[0093] First, the framework of this example can be referred to Figure 4 , including image and label data input module, image block mapping module, attribute feature embedding module, Transformer encoding module, attribute feature distillation module, attribute-guided recognition feature selection module, and face feature classification module.

[0094] During the model training phase, a face recognition model can be trained.

[0095] The image data and label information of the training data set are passed into the training system through the image and label data input module. The input image has a width of W, a height of H, and a number of channels of channel_dim. The label information includes the category information of the target face image and the attribute information of the target face image, including age, angle, blur, etc.

[0096] The image block mapping module divides the input image into blocks according to the set hyperparameter P (block size), which can be calculated according to the formula N = HW / P 2, and N image blocks are obtained. The output of this module has a dimension mapping layer that maps N image blocks to N token vectors of dimension embed_dim. The block division method can be to directly divide the original image data into blocks, or to use the convolution operator Conv2d operator. At this time, the parameter configuration of the convolution operator is set as Conv2d (in_channel = channel_dim, out_channel = embed_dim, kernel_size = P, stride = P, padding = False). The dimension mapping layer can be an operator such as a fully connected layer.

[0097] The attribute feature embedding module embeds M face attribute token feature vectors with dimension size embed_dim according to the M attributes contained in the training dataset. These M attributes will include age, angle, blur, etc.

[0098] After the N face token feature vectors and the M attribute token feature vectors obtained by the previous processing, before being input into the Transformer encoding module, it is necessary to add vectors expressing position information to these vectors, so that N+M position vectors of dimension embed_dim are required. The position vector can be a set of pre-set vectors or a set of learnable parameter vectors.

[0099] The N face token feature vectors with position information added are input to the Transformer encoding module together with the M attribute token feature vectors. In this module, they are first processed by the Norm layer and then sent to the multi-head attention layer. In the multi-head attention layer, since the query feature (Q), key feature (k), and value feature (V) all come from the same set of feature vectors, this set of feature vectors is the N face token vectors and the M attribute token feature vectors. In this way, after being processed by the multi-head attention layer, the M+N feature vectors output, each of which will fuse the information of the input N+M token feature vectors. These M+N feature vectors are then processed by the norm layer and the MLP layer, and finally output N+M feature vectors with a dimension of embed_dim. After the encoding module undergoes such processing many times, the final module outputs N+M feature vectors with a dimension of embed.

[0100] After the Transformer encoding module is processed, the N+M feature vectors are obtained, of which the first N are face block features and the last M are embedded face attribute features. These M embedded face attribute features will be distilled using an externally trained attribute model. This attribute teacher model can be a model built using a CNN network or a model using a ViT architecture. The objective function of the distillation can be: distill_ft =(1-α)L CE(ψ(Z s ),y)+ατ 2 KL(ψ(Z s / τ),ψ(Z t / τ)), or the formula

[0101] After distillation, the M facial attribute features and N facial block features are input to the attribute-guided recognition feature selection module. In this module, the M facial attribute features are used as query features Q, the N facial block features are used as key features K and value features V, and the formula cross_attention(Q,K,V)=Softmax(Q*K T After the cross attention processing shown in / √d)*V, M facial features with attribute adaptation are obtained.

[0102] The M facial features generated by M attributes are input to the facial feature classification module. This module fuses the M attribute-adaptive facial features to generate the final recognition features. In the training phase, the recognition features are further mapped to the categories of faces in the training data set, that is, they are classified into C categories based on the recognition features. The objective function of supervised classification can be the target loss function proposed by the ArcFace algorithm, such as the formula As shown in the figure. Among them, X represents the facial features extracted by the model for recognition, y is the category number corresponding to the input face image, s is a scale hyperparameter, and θ l is the angle between the lth column of the learnable classification parameter matrix and the input face feature X, m is an angle constraint hyperparameter greater than zero, and C is the number of face categories in the training data.

[0103] Finally, the overall objective function of the entire system is the sum of face classification loss and attribute distillation loss, as shown in formula L total =(1-λ)L arc +λL distill As shown in Figure 2, λ is a hyperparameter in the range of [0,1].

[0104] The recognition process for the image to be processed can be as follows:

[0105] The image data is passed into the system through the image and label data input module. Here, it is assumed that the width of the input image is W, the height is H, and the number of channels is channel_dim.

[0106] The data processing flow of the image block mapping module, attribute feature embedding module and Transformer encoding module is the same as the training process. The attribute feature distillation module is removed in the inference stage, and the N+M feature vectors output by the Transformer encoding module are directly output to the attribute-guided recognition feature selection module. The data processing in this module is also the same as the training process. The output M facial features are passed to the facial feature classification module. This module fuses the M attribute-adaptive facial features to generate the final recognition feature, which will be the final output feature.

[0107] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0108] Reference Figure 5 , shows a structural block diagram of an embodiment of a face image recognition device of the present invention, and the face image recognition device may specifically include the following modules:

[0109] An acquisition module 501 is used to acquire a face image to be recognized;

[0110] The first processing module 502 is used to map the face image to be recognized and generate a face image token vector sequence;

[0111] The second processing module 503 is used to perform cross-attention processing on the face image token vector sequence and the preset attribute token feature vector sequence to generate a face feature sequence;

[0112] A third processing module 504, configured to convert the facial feature sequence into recognition features;

[0113] The recognition module 505 is used to process the face image to be recognized based on the recognition feature.

[0114] In an optional embodiment of the present invention, the first processing module 502 includes:

[0115] A segmentation module, used for segmenting the face image to be recognized to generate multiple image blocks;

[0116] A mapping submodule, used for mapping the image block to generate a face image token vector;

[0117] The first combining submodule is used to combine the face image token vectors to generate a face image token vector sequence.

[0118] In an optional embodiment of the present invention, the preset attribute token feature vector sequence includes attribute token feature vectors corresponding to multiple different facial attributes, and the second processing module 503 includes:

[0119] An interaction submodule, used for interacting the face image token vector in the face image token vector sequence with the attribute token feature vector in the preset attribute token feature vector sequence to generate multiple face features;

[0120] The second combining submodule is used to combine the facial features to generate a facial feature sequence.

[0121] In an optional embodiment of the present invention, the second processing module 503 further includes:

[0122] An encoding submodule, used for position encoding the face image token vector and the attribute token feature vector;

[0123] The execution submodule is used to use the encoded face image token vector and the encoded attribute token feature vector to execute the step of interacting the face image token vector in the face image token vector sequence with the attribute token feature vector in the preset attribute token feature vector sequence to generate multiple face features.

[0124] In an optional embodiment of the present invention, the attribute token feature vector is determined by performing knowledge distillation on the annotated facial attribute information.

[0125] In an optional embodiment of the present invention, the third processing module 504 includes:

[0126] The fusion submodule is used to fuse the facial features to obtain recognition features.

[0127] In an optional embodiment of the present invention, the device further comprises:

[0128] A first training module, used to obtain a training image set and an initial model;

[0129] The second training module is used to train the initial model with the training image set to determine the face recognition model; the face recognition model is used to execute the step of mapping the face image to be recognized to generate a face image token vector sequence, and / or execute the step of cross-attention processing the face image token vector sequence and the preset attribute token feature vector sequence to generate a face feature sequence, and / or execute the step of converting the face feature sequence into recognition features.

[0130] The embodiment of the present invention obtains a face image to be identified; maps the face image to be identified to generate a face image token vector sequence; performs cross-attention processing on the face image token vector sequence and a preset attribute token feature vector sequence to generate a face feature sequence; converts the face feature sequence into identification features; and processes the face image to be identified based on the identification features. By making full use of the attribute information corresponding to the face image data, the face image token vector sequence and the preset attribute token feature vector sequence are cross-attention processed. The attributes in the preset attribute token feature vector sequence can correspond to age, blur, and angle, so that they are robust to these ages, blurs, and angles, and the obtained face feature sequence is associated with these attributes. Therefore, when faced with problems such as large age spans, blurred images, and large face angles, face images can still be correctly identified, thereby improving the accuracy of face image recognition.

[0131] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0132] Reference Figure 6 , an embodiment of the present invention further provides an electronic device, including:

[0133] A processor 601 and a storage medium 602, wherein the storage medium 602 stores a computer program executable by the processor 601, and when the electronic device is running, the processor 601 executes the computer program to perform the face image recognition method as described in any one of the embodiments of the present invention.

[0134] The facial image recognition method comprises:

[0135] Obtain a face image to be recognized;

[0136] Mapping the face image to be recognized to generate a face image token vector sequence;

[0137] Performing cross-attention processing on the face image token vector sequence and the preset attribute token feature vector sequence to generate a face feature sequence;

[0138] Converting the facial feature sequence into recognition features;

[0139] The face image to be identified is processed based on the identification features.

[0140] Optionally, the step of mapping the face image to be identified to generate a face image token vector sequence includes:

[0141] Segmenting the face image to be recognized to generate multiple image blocks;

[0142] Mapping the image block to generate a face image token vector;

[0143] The face image token vectors are combined to generate a face image token vector sequence.

[0144] Optionally, the preset attribute token feature vector sequence includes attribute token feature vectors corresponding to multiple different facial attributes, and the step of performing cross-attention processing on the facial image token vector sequence and the preset attribute token feature vector sequence to generate a facial feature sequence includes:

[0145] Interacting a face image token vector in a face image token vector sequence with an attribute token feature vector in a preset attribute token feature vector sequence to generate a plurality of face features;

[0146] The facial features are combined to generate a facial feature sequence.

[0147] Optionally, the step of performing cross-attention processing on the facial image token vector sequence and the preset attribute token feature vector sequence to generate a facial feature sequence further includes:

[0148] Position encoding the face image token vector and the attribute token feature vector;

[0149] The encoded face image token vector and the encoded attribute token feature vector are used to perform the step of interacting the face image token vector in the face image token vector sequence with the attribute token feature vector in the preset attribute token feature vector sequence to generate multiple face features.

[0150] Optionally, the attribute token feature vector is determined by performing knowledge distillation on labeled facial attribute information.

[0151] Optionally, the step of converting the facial feature sequence into recognition features includes:

[0152] The facial features are fused to obtain identification features.

[0153] Optionally, the method further comprises:

[0154] Get the training image set and initial model;

[0155] The initial model is trained using the training image set to determine a face recognition model; the face recognition model is used to execute the step of mapping the face image to be recognized to generate a face image token vector sequence, and / or execute the step of cross-attention processing the face image token vector sequence and a preset attribute token feature vector sequence to generate a face feature sequence, and / or execute the step of converting the face feature sequence into recognition features.

[0156] The embodiment of the present invention obtains a face image to be identified; maps the face image to be identified to generate a face image token vector sequence; performs cross-attention processing on the face image token vector sequence and a preset attribute token feature vector sequence to generate a face feature sequence; converts the face feature sequence into identification features; and processes the face image to be identified based on the identification features. By making full use of the attribute information corresponding to the face image data, the face image token vector sequence and the preset attribute token feature vector sequence are cross-attention processed. The attributes in the preset attribute token feature vector sequence can correspond to age, blur, and angle, so that they are robust to these ages, blurs, and angles, and the obtained face feature sequence is associated with these attributes. Therefore, when faced with problems such as large age spans, blurred images, and large face angles, face images can still be correctly identified, thereby improving the accuracy of face image recognition.

[0157] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the processor.

[0158] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0159] Reference Figure 7 The embodiment of the present invention further provides a computer-readable storage medium 701, on which a computer program is stored. When the computer program is executed by a processor, the face image recognition method as described in any one of the embodiments of the present invention is executed.

[0160] The facial image recognition method comprises:

[0161] Obtain a face image to be recognized;

[0162] Mapping the face image to be recognized to generate a face image token vector sequence;

[0163] Performing cross-attention processing on the face image token vector sequence and the preset attribute token feature vector sequence to generate a face feature sequence;

[0164] Converting the facial feature sequence into recognition features;

[0165] The face image to be identified is processed based on the identification features.

[0166] Optionally, the step of mapping the face image to be identified to generate a face image token vector sequence includes:

[0167] Segmenting the face image to be recognized to generate multiple image blocks;

[0168] Mapping the image block to generate a face image token vector;

[0169] The face image token vectors are combined to generate a face image token vector sequence.

[0170] Optionally, the preset attribute token feature vector sequence includes attribute token feature vectors corresponding to multiple different facial attributes, and the step of performing cross-attention processing on the facial image token vector sequence and the preset attribute token feature vector sequence to generate a facial feature sequence includes:

[0171] Interacting a face image token vector in a face image token vector sequence with an attribute token feature vector in a preset attribute token feature vector sequence to generate a plurality of face features;

[0172] The facial features are combined to generate a facial feature sequence.

[0173] Optionally, the step of performing cross-attention processing on the facial image token vector sequence and the preset attribute token feature vector sequence to generate a facial feature sequence further includes:

[0174] Position encoding the face image token vector and the attribute token feature vector;

[0175] The encoded face image token vector and the encoded attribute token feature vector are used to perform the step of interacting the face image token vector in the face image token vector sequence with the attribute token feature vector in the preset attribute token feature vector sequence to generate multiple face features.

[0176] Optionally, the attribute token feature vector is determined by performing knowledge distillation on labeled facial attribute information.

[0177] Optionally, the step of converting the facial feature sequence into recognition features includes:

[0178] The facial features are fused to obtain identification features.

[0179] Optionally, the method further comprises:

[0180] Get the training image set and initial model;

[0181] The initial model is trained using the training image set to determine a face recognition model; the face recognition model is used to execute the step of mapping the face image to be recognized to generate a face image token vector sequence, and / or execute the step of cross-attention processing the face image token vector sequence and a preset attribute token feature vector sequence to generate a face feature sequence, and / or execute the step of converting the face feature sequence into recognition features.

[0182] The embodiment of the present invention obtains a face image to be identified; maps the face image to be identified to generate a face image token vector sequence; performs cross-attention processing on the face image token vector sequence and a preset attribute token feature vector sequence to generate a face feature sequence; converts the face feature sequence into identification features; and processes the face image to be identified based on the identification features. By making full use of the attribute information corresponding to the face image data, the face image token vector sequence and the preset attribute token feature vector sequence are cross-attention processed. The attributes in the preset attribute token feature vector sequence can correspond to age, blur, and angle, so that they are robust to these ages, blurs, and angles, and the obtained face feature sequence is associated with these attributes. Therefore, when faced with problems such as large age spans, blurred images, and large face angles, face images can still be correctly identified, thereby improving the accuracy of face image recognition.

[0183] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0184] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0185] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0186] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0187] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0188] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0189] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0190] The above is a detailed introduction to a facial image recognition method, a facial image recognition device, an electronic device and a storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a general technician in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A facial image recognition method, characterized in that: include: Obtain a face image to be recognized; Mapping the face image to be recognized to generate a face image token vector sequence; Performing cross-attention processing on the face image token vector sequence and the preset attribute token feature vector sequence to generate a face feature sequence; Converting the facial feature sequence into recognition features; The face image to be identified is processed based on the identification features.

2. The method according to claim 1, characterized in that The step of mapping the face image to be identified to generate a face image token vector sequence comprises: Segmenting the face image to be recognized to generate multiple image blocks; Mapping the image block to generate a face image token vector; The face image token vectors are combined to generate a face image token vector sequence.

3. The method according to claim 2, characterized in that The preset attribute token feature vector sequence includes attribute token feature vectors corresponding to multiple different facial attributes, and the step of performing cross-attention processing on the facial image token vector sequence and the preset attribute token feature vector sequence to generate a facial feature sequence includes: Interacting a face image token vector in a face image token vector sequence with an attribute token feature vector in a preset attribute token feature vector sequence to generate a plurality of face features; The facial features are combined to generate a facial feature sequence.

4. The method according to claim 3, characterized in that: The step of performing cross-attention processing on the face image token vector sequence and the preset attribute token feature vector sequence to generate a face feature sequence also includes: Position encoding the face image token vector and the attribute token feature vector; The encoded face image token vector and the encoded attribute token feature vector are used to perform the step of interacting the face image token vector in the face image token vector sequence with the attribute token feature vector in the preset attribute token feature vector sequence to generate multiple face features.

5. The method according to claim 3, characterized in that: The attribute token feature vector is determined by performing knowledge distillation on the annotated face attribute information.

6. The method according to claim 3, characterized in that The step of converting the facial feature sequence into recognition features comprises: The facial features are fused to obtain identification features.

7. The method according to claim 1, characterized in that The method further comprises: Get the training image set and initial model; The initial model is trained using the training image set to determine a face recognition model; the face recognition model is used to execute the step of mapping the face image to be recognized to generate a face image token vector sequence, and / or execute the step of cross-attention processing the face image token vector sequence and a preset attribute token feature vector sequence to generate a face feature sequence, and / or execute the step of converting the face feature sequence into recognition features.

8. A facial image recognition device, characterized in that: include: An acquisition module, used to acquire a face image to be recognized; A first processing module, used for mapping the face image to be identified to generate a face image token vector sequence; A second processing module is used to perform cross-attention processing on the face image token vector sequence and the preset attribute token feature vector sequence to generate a face feature sequence; A third processing module, used for converting the facial feature sequence into recognition features; The recognition module is used to process the face image to be recognized based on the recognition feature.

9. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the face image recognition method as claimed in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the face image recognition method according to any one of claims 1 to 7 are implemented.

Images