A small sample scene based on double flow prototype occluded face recognition method

Abstract

In order to solve the problem of occlusion recognition in small sample scene of face pictures, the application discloses a double-flow prototype learning method for pictures with occlusion in small sample scene. Unlike the class prototype learning method which considers optimization from the perspective of the support set or the query set, the framework of the application combines the two, which can learn the corresponding adaptive sample weight for the sample pictures in the support set. In this way, the uneven picture quality caused by occlusion in the support set is considered, and the weight of the occlusion sample can be reduced in the flow centered on the support set. In addition, the feature alignment module is used in the prototype construction, so that the middle-level feature representation related to the query set picture is considered, and the consistency constraint of the similarity between the high-level feature representation makes the prototype learned in the flow centered on the query set more reliable. The application improves the accuracy of occluded face recognition of new subjects, and can provide help for practical application.

Description

TECHNICAL FIELD

[0001] The technology belongs to the field of artificial intelligence and picture classification, and particularly relates to a double-flow prototype learning method for pictures with occlusions in a small sample scenario. BACKGROUND

[0002] In recent years, deep learning technology has made great achievements in recognizing face pictures with occlusions, which is mainly due to the following two factors: a large-scale, high-quality face repository and limited occlusion scene patterns (for example, the occluded face dataset is artificially synthesized by directly adding occlusions such as sunglasses and scarves to existing face pictures). However, in reality, for a new subject that a machine has never learned before, the number of labeled face pictures available for machine learning is very small. This makes it difficult for the model trained on the face repository to quickly learn from the small number of samples of the new subject based on prior knowledge like humans, and may require retraining by adding the extremely limited face pictures of the new subject to the training data, which consumes a lot of computing time. Moreover, the artificially synthesized occlusions are very different from the natural occlusions in real life, which usually come from: capturing non-frontal faces, facial shadows caused by extreme light, and partial occlusions caused by masks, etc. Therefore, it is of great significance to design a method that can recognize faces in the case of possible occlusions in a small sample scenario.

[0003] The problem of face recognition under occlusions in a small sample scenario is called FSFRO (Few-Shot Face Recognition under Occlusions), which can be modeled as a picture classification task in the field of small sample learning, where a subject is regarded as a class; and the occlusions in the face pictures are represented as different degrees of feature jitter in the feature representation learning process of the model. The impact of this feature jitter will be more severe in a small sample scenario.

[0004] The picture classification methods of small sample learning mainly include three categories: data-based, metric-based and algorithm-based. The data-based method uses priori to do data enhancement on a small amount of training data to increase the sample size during training. The algorithm-based method trains the network model for classification by using a small amount of labeled samples, and then optimizes the network model in the test stage by using a new task. The metric-based method usually learns the class representation by using a small amount of labeled samples, and then determines the class of the object to be classified according to the relationship between the feature representation of the object to be classified and the class representation. For example, the Prototypical Networks calculates the fixed Euclidean distance between the feature representation of the query sample and the average feature representation (class prototype) of all samples of each class in the support set. In particular, to solve the feature jitter problem in the feature representation learning process, most solutions are to reduce the intra-class difference of the feature representation of the same class of objects and correct the class prototype, so that the extracted feature representation is robust to the occlusion scene. However, they often only consider the relationship between the pictures in the support set from the same class or only consider the relationship between the query set and the pictures in the support set when constructing the class prototype.

[0005] Although many metric-based methods can be used to solve the small sample occluded face picture classification problem, however, the existing methods have the following shortcomings:

[0006] 1. Most of the existing methods only consider the relationship between the pictures in the support set from the same class or only consider the relationship between the query set and the pictures in the support set when learning the class prototype.

[0007] 2. In the feature representation learning process, the existing methods mostly regard the pictures with occlusion as low-quality samples, which is not necessarily applicable when the query sample belongs to the occluded sample. SUMMARY

[0008] The present application provides a dual-flow prototype learning method for occluded face picture recognition in a small sample scenario to overcome the shortcomings of the prior art. The purpose of the invention is:

[0009] 1. A general dual-flow framework is proposed to learn and optimize the class prototype under the condition of small samples, which considers the correlation between the pictures in the support set and their relationship with the query sample.

[0010] 2. The contribution of the occluded face picture in the formation of the class prototype is adaptively adjusted from the support set and the query set.

[0011] The technical solution of this invention is an occluded face recognition method based on a two-stream prototype in small sample scenarios, which is implemented by a two-stream prototype learning network. We name it: Two-Stream Prototype Learning Network (TSPLN), which includes the following steps:

[0012] Step S1: Pre-train a transferable similarity relationship network using the dataset, which consists of non-overlapping base classes. and new categories composition, , ;

[0013] Step S2, construct an N-way K-shot few-shot learning task:

[0014] In the few-shot learning setting, a sampled N-way K-shot training or testing task is defined as { , },in To support the collection, For query sets; random sampling from base classes or new classes. N Categories And then from N Sampling of each class K The support set consists of 10 samples. Q The query set consists of 10 samples;

[0015] Step S3: Extract image features using a feature embedding network. The obtained features will be used for subsequent prototype learning of the network.

[0016] Step S4: Calculate the similarity of images within the support set based on the pre-trained similarity relation network;

[0017] Step S5: Implement category prototype learning centered on support sets based on a transferable similarity relationship network;

[0018] Step S6: Design a feature alignment module to calculate the feature alignment score between the query set and the support set as the relevance.

[0019] Step S7, introduce the consistency loss of KL divergence;

[0020] Step S8: Implement query set-centric category prototype learning based on the feature alignment module;

[0021] Step S9: During the training phase, predict the label of the query sample in a small sample learning task.

[0022] Step S10: In the prediction phase, the query samples in the small sample task of the new type of data are predicted.

[0023] Further, the specific implementation of step S5 is as follows:

[0024] In forming the prototype of the class , the feature representation corresponding to the occluded sample picture is given a small weight, and the feature representation weight of each support set picture is from the sample similar to other samples belonging to the class The average of the similarity is calculated as follows:

[0025] ,

[0026] Among them, i and j represent the index, so as to obtain the weighted class representation of the class in the data stream centered on the support set instead of simply averaging the features, which can be represented as follows:

[0027] ,

[0028] Among them represents the feature embedding network, which outputs the feature representation of the object ; represents the class label of the sample .

[0029] Further, step S6 calculates the feature alignment score between the query set and the support set by designing a feature alignment module between the query sample and the support sample, and the specific implementation is as follows:

[0030] The feature alignment module first selects the feature map at the intermediate level of the feature embedding network to a certain extent by a feature down-sampling convolution block, and removes some redundant feature information; Then, the new feature map of each sample in the support set and the new feature map of each sample in the query set can be obtained, and then the similarity between pixel pairs and is calculated and the sum of the similarities of all pixel points is added as the alignment score of the intermediate-level features of each query sample and the support set sample , which can be represented as follows:

[0031] ,

[0032] Among them, and represent the feature map and​ where the meaning of H, W is the length and width of the feature map.

[0033] Further, the KL divergence loss in step S7 is calculated as follows:

[0034]

[0035] where, and represent the high-level feature representation of the whole samples in query set and support set, respectively. denotes the set of all alignment scores between the query set and the support set.

[0036] Further, in step S8, for each class , the learned query-centered class prototype is obtained according to the weight of the prototype, and the specific calculation method is as follows:

[0037] ,

[0038] where, denotes the class label of the sample , and denotes the index, and the alignment score provides the weight information when forming the class prototype, denotes the alignment score between the th support set image in the query set and the class , and the query sample will be classified into the class to which the nearest class prototype belongs.

[0039] Further, the loss function for optimizing the network structure in the training stage is set as follows:

[0040] The classification loss of the support set-centered stream and the classification loss of the query set-centered stream are as follows:

[0041] ,

[0042] ,

[0043] where, is the true label, denotes the sample index; is an indicator function: when is true, otherwise it is 0; and denote the probability that each query image is predicted as class in the support set-centered stream and the query set-centered stream, respectively.

[0044] Final total loss function Prototype classification loss for support set centered stream Prototype classification loss for query set centered stream And the combination of the KL loss appended when optimizing the alignment score;

[0045] + ,

[0046] Where, Is a balanced hyper-parameter.

[0047] Further, the feature embedding network is ResNet50.

[0048] Unlike the class prototype learning method which only considers optimization from the perspective of the support set or the query set, the framework of the present application combines the two, which can learn the corresponding adaptive sample weight for the sample picture in the support set. In this way, both the uneven picture quality due to occlusion in the support set and the weight of the occlusion sample in the support set centered stream are considered; the feature alignment module is used when constructing the prototype, so that the middle-level feature representation related to the query set picture is considered, and the consistency constraint of the similarity between the high-level feature representation makes the prototype learned in the query set centered stream more reliable. The present application solves the problem of recognizing occluded faces in small sample scenarios, improves the accuracy of recognizing occluded faces of new subjects, and can provide help for practical applications. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 Is the overall flowchart of the present application. It lists the main steps included in the framework of the application.

[0050] Figure 2 Is the double-stream prototype learning network structure diagram in the present application. DETAILED DESCRIPTION

[0051] The preferred embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0052] As shown in Figure 1 And Figure 2 The present application provides a double-stream prototype based occluded face recognition method in a small sample scenario, which models the problem of recognizing faces under occlusion in a small sample scenario as a picture classification task in the field of small sample learning, and proposes a general double-stream framework to learn and optimize the class prototype under the condition of small samples, while considering the correlation between the support set pictures and their relationship with the query sample; Specifically, it includes the following steps:

[0053] S1. Pre-training a transferable similarity relation network:

[0054] Our dataset consists of mutually exclusive base classes and new classes , with the data of new classes being unseen during training. Since transferable pairwise semantic similarity can bridge the gap between base classes and new classes, we introduce a transferable similarity relation network (TSRN). The TSRN aims to learn similarity relations that can generalize to new classes to facilitate subsequent prototype learning. The main body of the TSRN is composed of three layers of "linear - batch normalization - activation function (Relu)" stacked together. The last layer of the TSRN is a classification head function that completes binary classification, outputting a probability distribution belonging to labels 0 and 1 respectively and comparing it with the true label value. Among them, the true label follows the principle that the picture comes from the same class, the label is 1, otherwise it is 0. During pre-training, the TSRN takes sufficient data in the base class as input, and it learns the nonlinear relationship between the pair of deep feature representations. Once the training is complete, the TSRN can give the pairwise similarity of any two input pictures. Although we expect similarity to be class-independent knowledge, there is still a certain degree of domain gap between base classes and new classes. Therefore, we use artificially added Gaussian noise in the new class data to reduce the domain gap during training of the TSRN. This does not contradict our second stage of small sample learning problem settings, because clean new classes are still not visible during the entire training period. The available labels for the TSRN are similar / not similar (pictures from the same class are similar, and pictures from different classes are not similar), and the model does not know whether they come from the base class or the new class.

[0055] S2. Construct an N-way K-shot small sample learning task:

[0056] Under the setting of small sample learning, a sampled N-way K-shot training / testing task is defined as , } where is the support set and is the query set. The construction process is as follows: in an episode, we randomly sample N classes from the base class / new class, then sample N samples from each of the K classes to form the support set, and Q samples to form the query set, so there areN K samples, query set has N Q In the support set, the number of samples per class, which is to simulate the scenario of a small number of new classes in the meta-test.

[0057] S3. Extract image features using feature embedding network:

[0058] In order to get the feature embedding of the image, we use ResNet50 with CNN structure as the feature embedding network. Given a N-way K-shot small sample task support set, then contains N K samples of support set and contains N Q query set is the input of the feature embedding network, after the learning of the backbone, finally for each sample will get a high-dimensional feature embedding (2048 dimensions). In fact, it is to learn a mapping from sample to feature ). The obtained feature will be used for subsequent prototype learning of the entire network.

[0059] S4. Calculate the similarity of pictures in the support set based on the pre-trained similarity relationship network:

[0060] Given a N-way K-shot small sample task support set, we will send the feature embedding of pictures belonging to class K into TSRN, and will get the semantic similarity matrix of pictures in the class K .

[0061] S5. Implement support-centered prototype learning (Support-centered Stream) based on the transferable similarity relationship network:

[0062] When only considering the relationship between pictures in the support set, we observe that the occluded face picture and the unoccluded face picture are not similar, and the picture with occlusion will have a negative impact on the class prototype. In order to reduce this impact, a direct and intuitive method is to make the feature of the occluded picture contribute less to the subsequent formation of the class prototype, and this contribution degree is consistent with the similarity of other unoccluded samples in the same class. In the formation of the class ​When we are learning the prototype of a class, we need to give small weights to the feature representation of the occluded samples. The weight of the feature representation of each support set image is derived from the similarity of the sample to other samples in the same class and the average value of the similarity is calculated as follows:

[0063] ,

[0064] With the weights, we can get the weighted class representation of the class in the data stream centered on the support set, rather than the simple average feature. It can be expressed as follows:

[0065] ,

[0066] where denotes the feature embedding function, and we choose the feature embedding network ResNet50, which outputs the feature representation of the object as described in S3; denotes the class label of the sample .

[0067] S6. Calculate the feature alignment score between the query set and the support set as the relevance:

[0068] For our problem FSFRO, it is likely that the query image is exactly an occluded image, and the images in the support set that are occluded will play a positive role in the classification task, and the weight should not be reduced because of low quality. The prototype learned only by the support set is difficult to maintain optimal consistency when facing different query samples. Therefore, we designed a feature alignment module between the query sample and the support sample to select the most relevant sample features to the query sample.

[0069] In addition, considering that the middle-level feature representation of the image is class-independent, the middle layer of the backbone (i.e., ResNet50) that extracts the features is equivalent to an implicit memory block that stores knowledge that can be shared and transferred between base classes and new classes. The input of the feature alignment module is the feature map of the third layer of the backbone of the query sample and the support set images.

[0070] The feature alignment module first learns a feature down-sampling convolutional block to select the middle-level feature map to some extent and remove some redundant feature information. Then, we can get the new feature map of each sample in the support set and the new feature map of each sample in the query set Then, as shown in Feature Alignment, we calculate the similarity between each pixel of the query sample and the support sample and add all the similarities as the alignment score of the middle-level feature between the query sample and the support sample Figure 2 The alignment score of the middle-level feature between the query sample and the support sample can be represented as follows:

[0071]

[0072] wherein, and represent the pixels in the feature maps and , and H and W represent the height and width of the feature map, respectively.

[0073] S7. Introduce the consistency loss of KL divergence:

[0074] For a query set with N Q query samples, it will get N NQ alignment scores after alignment with an N-way K-shot support set. However, we cannot completely rely on the alignment score because the query picture may have a high alignment score with the support set picture of the non-query class due to some background or hair color in the picture, resulting in misclassification. Therefore, the KL divergence loss is introduced to constrain the alignment score of the high-level feature representation and the middle-level feature representation, so that they remain consistent. In fact, the high-level feature representation is used as a soft label because the high-level feature representation is connected to the classification head and has more discriminative power. The loss calculation of the KL divergence is as follows:

[0075]

[0076] wherein, and represent the high-level feature representations of all samples in the query set and the support set, respectively, ​​​​The set of all alignment scores of the whole query set and support set, where the high-level features are the output of the fourth layer (the last layer) of the ResNet50 (also called Backbone) after average pooling, ResNet50 is composed of 4 stacked Residual blocks, and the fourth layer is the last Residual block, so it is high-level, and the middle-level features are the output of the third layer.

[0077] S8. Query-centered stream prototype learning is realized based on the feature alignment module:

[0078] Similar to the support-centered stream prototype learning process in S5, the alignment score provides weight information when forming class prototypes. We can learn the query-centered class prototype of each class according to the weight of the prototype . The specific calculation method is as follows:

[0079] ,

[0080] wherein, represents the alignment score of the i-th support set image in the query set and the class . The query sample is classified into the class to which the nearest class prototype belongs.

[0081] S9. Predict the label of the query sample in a small sample learning task in the training phase:

[0082] The training model is constructed by calculating the classification loss of the feature representation of the query sample and the class prototype using the Euclidean distance. In training, we calculate the distance between the query sample and the learned prototype in each data stream, and use the softmax function to obtain the classification probability, and then use the cross-entropy function to classify. The classification loss of the support-centered stream and the classification loss of the query-centered stream are as follows:

[0083] ,

[0084] ,

[0085] wherein, is the true label; is an indicator function: when is true, otherwise 0; and ​respectively represent the probability that each query image is predicted as the class in the support set centered stream and the query set centered stream. When optimizing the query set centered stream, as mentioned in S6, to assist the feature alignment module to get as correct alignment scores as possible in order to learn the optimized prototypes, we introduce a KL divergence loss. Finally, the total loss function is the combination of the prototype classification loss of the support set centered stream , the prototype classification loss of the query set centered stream , and the additional KL loss when optimizing the alignment scores.

[0086] + ,

[0087] where, is a balanced hyper-parameter.

[0088] In addition, the training stage adopts an episode training strategy, in which the steps of S2-S9 are repeated, which is the training of different small sample tasks sampled multiple times.

[0089] S10. The test stage predicts the query samples in the small sample task of the new class data:

[0090] As described in the background, we actually convert the occluded face recognition problem in the small sample scenario into a face image classification problem, and each person is a class, and then the new identity face is a new class in the small sample scenario. Therefore, when testing after the model is trained, first, a small sample learning task is also sampled according to the training, but the sampling is performed in the new class data. Among them, each new class only has K labeled face images and the images may be occluded. Then, we use the class prototypes learned through the two data streams as class representations. The query samples of the new identity (that is, the new class) in the task make class predictions according to the two sets of class prototypes respectively to complete classification, that is, face recognition of the new identity.

[0091] The classification accuracy is used to verify the effectiveness of the model. The classification accuracy (Accuracy) is a commonly used evaluation index of image classification method, that is, the proportion of samples classified correctly to the total number of samples in the class. Test result analysis: after multiple tests, for the face data set with occlusion, the accuracy exceeds the current most advanced small sample learning method under the small sample learning setting.

[0092] We evaluate our method on two publicly available and widely used face datasets: CASIA-WebFace and RMFRD, each containing many subjects. The RMFRD and CASIA-WebFace datasets are designed for general supervised face recognition. Therefore, we utilize part of the subjects in our experiments and re-split them for few-shot learning. Specifically, we randomly select 200 subjects from each dataset. Then we split the selected 200 subjects into 150 and 50; respectively as the base classes for meta-training and new classes for meta-testing. In all datasets, images are resized to 84x84.

[0093] RMFRD is a large-scale real masked face dataset crawled from internet resources, containing 525 subjects. Since there are some subjects in the dataset without occluded face images, there are only 460 subjects available in our experiments. The CASIA-WebFace dataset is a collection of face images of over 10,000 subjects from the IMDb website in unconstrained environments. The face images in CASIA-WebFace contain rich variations in pose, facial accessories, illumination, resolution, etc. These variations fully reflect the face occlusion patterns in the wild.

[0094] Table 1 is the comparative results (accuracy %) of the present application in the 5-way 3-shot and 5-way 5-shot small sample settings on the two occluded face datasets RMFRD and CASIA-WebFace.

[0095] Table 1

[0096]

[0097] Table 1 shows the classification accuracy of different methods on all datasets. Including the implementation of the present technical solution and other comparative methods, we have three observations:

[0098] (1) Our TSPLN achieves the state-of-the-art performance in the 5-way 3-shot / 5-shot setting on two face datasets with rich occlusion. Compared with the strongest baseline RENet, it improves by 3.19% and 5.33% on the dataset RMFRD and CASIA-WebFace in the 3-shot setting, and by 1.84% and 3.46% on the dataset RMFRD and CASIA-WebFace in the 5-shot setting.

[0099] (2) Our method greatly outperforms directly introducing ArcFace loss into the baseline of common few-shot learning, which indicates that ArcFace loss cannot help to solve the FSFRO problem. In contrast, our method can deal with both the few-shot and the occlusion challenges in a general manner without special consideration of the way of face distance measurement.

[0100] (3) The results in Table 1 show that our method performs less degradation as the shot number decreases. We assume that the occluded face recognition in the few-shot setting is a class-level feature jitter. When the shot number decreases in a few-shot task, this feature jitter is more likely to occur, so other baseline methods have a larger performance degradation in the 3-shot setting compared to the 5-shot setting. Our proposed solution degrades less as the shot number decreases, with a 2.37% decrease in RMFRD.

[0101] In summary, our method can perform better in real-world scenarios with fewer images of new subjects and rich natural occlusions in the images.

[0102] The specific embodiments described herein are merely illustrative of the spirit of the present application. Those skilled in the art can make various modifications or supplements to the described specific embodiments or use similar ways to replace, but will not deviate from the spirit of the present application or exceed the scope defined by the appended claims.

Claims

1. A method for occluded face recognition based on double-flow prototype in small sample scenes, characterized in that: The face recognition problem of small samples under occlusion is modeled as a picture classification task in the field of small sample learning, and a general double-flow framework is proposed to learn and optimize the class prototype under the condition of small samples. The correlation between support set pictures and their relationship with query samples are considered, which can be generalized to new identity face pictures containing only a small amount of labeled pictures, and there are different degrees of occlusion in the pictures. Specifically, the following steps are included: Step S1, pre-training a transferable similarity relation network with a dataset composed of base classes C base and new classes C novel , Step S2, construct an N-way K-shot small sample learning task: In the setting of few-shot learning, a sampled N-way K-shot training or testing task is defined as {S N , Q N} where S N is the support set and Q N is the query set; N classes C N are randomly sampled from the base classes or new classes, then K samples are sampled from each of the N classes to form the support set, and Q samples are sampled from each of the N classes to form the query set. Step S3, use the feature embedding network to extract image features, and the obtained features will be used for subsequent prototype learning of the network; Step S4, calculate the similarity of the pictures in the support set based on the pre-trained similarity relationship network; Step S5, realize class prototype learning based on the support set by using the transferable similarity relationship network; Step S6, design a feature alignment module to calculate the feature alignment score between the query set and the support set as the correlation; Step S7, introduce the consistency loss of KL divergence; Step S8, realize class prototype learning based on the query set by using the feature alignment module; Step S9, predict the label of the query sample in a small sample learning task in the training stage; Step S10, predict the query sample in the small sample task of the new class data in the prediction stage.

2. The occluded face recognition method based on double-flow prototypes in a small sample scene according to claim 1, wherein: The specific implementation of step S5 is as follows: In forming the prototype of class C n , the feature representation corresponding to the occluded sample picture is given a small weight, and the feature representation weight of each support set picture comes from the similarity M of the sample n to other samples belonging to class C , and the average value of the similarity M i,j is calculated as follows: where i and j represent indices, j≠i, to obtain a weighted class representation of class C n in the data stream centered on the support set rather than a simple average feature, may be represented as follows: where f(·) denotes a feature embedding network that characterizes the features of an output object ; denotes the class label of a sample .

3. The occluded face recognition method based on double-flow prototypes in a small sample scene according to claim 1, characterized in that: Step S6 designs a feature alignment module between the query sample and the support sample to calculate the feature alignment score between the query set and the support set, and the specific implementation is as follows: The feature alignment module firstly removes some redundant feature information by a feature down-sampling convolution block to a certain extent to select the feature map of the intermediate level of the feature embedding network; then, the new feature map of each sample of the support set and the new feature map of each sample of the query set can be obtained respectively and query set Then, the similarity of each pixel pair is calculated and The similarity of all pixel points is added as the alignment score S of the intermediate level features of each query sample and the support set sample align , which can be expressed as follows: where p m and p n represent the pixel points in the feature map and , and the meaning of H, W is the length and width of the feature map.

4. The occluded face recognition method based on double-flow prototypes in a small sample scene according to claim 1, characterized in that: The loss calculation of KL divergence in step S7 is as follows: wherein, and represent the high-level feature representation of the whole samples of the query set and the whole samples of the support set, respectively, refers to the set of all alignment scores of the whole query set and the whole support set.

5. The occluded face recognition method based on double-flow prototype in small sample scene according to claim 1, characterized in that: In step S8, the prototype of each category C n is obtained by weighting the learned query set centered category prototypes The specific calculation is as follows: wherein, represents a class label of a sample i represents an index, the alignment score provides weight information when forming class prototypes, represents an alignment score of the i-th support set image in all samples and class C n in the query set, the query sample will be classified into the class to which the nearest class prototype belongs.

6. The occluded face recognition method based on double-flow prototypes in a small sample scene according to claim 1, characterized in that: The loss function used to optimize the network structure in the training stage is set as follows: Classification loss for stream centered on support set Classification loss for stream centered on query set respectively as follows: where y i is the true label, i denotes the sample index; is an indicator function: 1 when 0 otherwise; and denote the probability that each query image is predicted as class j in the support set centered flow and the query set centered flow, respectively; Finally, the total loss function L is a combination of the prototype classification loss for the query set centered flow the prototype classification loss for the query set centered flow and the additional KL loss when optimizing the alignment score Where γ≥0 is a balance hyperparameter.

7. The method of claim 1, wherein the method is a method of occluded face recognition based on double-flow prototypes in a small sample scenario. The feature embedding network is ResNet50.

Images