Method for training face recognition model, face recognition method and storage medium

Abstract

The present disclosure relates to a method for training a face recognition model, a method for recognizing a face, and a storage medium. According to one embodiment of the present disclosure, the method for training a face recognition model comprises: determining a global feature similarity total loss, a key point position deviation total loss, and a classification total loss for a face sample image set by summation; determining a weighted loss by weighted summation of the global feature similarity total loss, the key point position deviation total loss, and the classification total loss; determining whether the weighted loss converges or is less than a predetermined threshold; and optimizing the face recognition model by adjusting parameters of the face recognition model in a case where the determination result is "no". The method and the storage medium of the present disclosure have at least the beneficial effect of improving the face recognition accuracy for face images with occlusions.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to image processing, and more particularly, to a method for training a face recognition model, a method for face recognition, and a storage medium. BACKGROUND

[0002] With the improvement of computer computing power, the speed and accuracy of face recognition have reached a practical application level. However, when there is a cover on the face that covers part of the face, the face captured by the camera is a face image with occlusion. Due to the existence of the cover, the recognition accuracy of the face recognition model decreases. Therefore, there is a challenge to recognize such face images with occlusion. SUMMARY

[0003] A brief summary of the disclosure will be given in the following to provide a basic understanding of some aspects of the disclosure. It should be understood that this summary is not an exhaustive overview of the disclosure. It is not intended to determine the key or important parts of the disclosure, nor to limit the scope of the disclosure. Its purpose is only to give some concepts in a simplified form as a prelude to the more detailed description discussed later.

[0004] According to one aspect of the present disclosure, a method for training a face recognition model is provided, comprising: for each face sample image pair including an unoccluded image and an occluded image in a face sample image set: determining a global feature similarity loss of the face sample image pair based on a global feature pair of the face sample image pair; determining a key point position deviation loss of the face sample image pair based on an n key point position pair of the face sample image pair; and determining a classification loss of the face sample image pair based on a fusion feature pair of the face sample image pair; determining a global feature similarity total loss, a key point position deviation total loss and a classification total loss for the face sample image set by summation; determining a weighted loss by weighted summation of the global feature similarity total loss, the key point position deviation total loss and the classification total loss; determining whether the weighted loss converges or is less than a predetermined threshold; and in the case of determining that the result is "no", optimizing the face recognition model by adjusting parameters of the face recognition model; wherein the unoccluded image and the occluded image included in each face sample image pair are an unoccluded face sample image and an occluded face sample image of the same person; each fusion feature in the fusion feature pair of the face sample image pair is a feature obtained by fusing the global feature and the local feature of the corresponding face sample image; and the local feature of the corresponding face sample image is associated with an unoccluded face local area.

[0005] According to an aspect of the present disclosure, a method for face recognition is provided, comprising: training a face recognition model using the aforementioned method for training a face recognition model; and performing face recognition on a face image to be recognized using the trained face recognition model.

[0006] According to an aspect of the present disclosure, a method for training a face recognition model is provided. The method comprises: for each face sample image pair comprising an unoccluded image and an occluded image in a face sample image set: determining a global feature similarity loss of the face sample image pair based on a global feature pair of the face sample image pair; determining a key point position deviation loss of the face sample image pair based on an N key point position pair of the face sample image pair; and determining a classification loss of the face sample image pair based on a fusion feature pair of the face sample image pair; determining a global feature similarity total loss, a key point position deviation total loss and a classification total loss for the face sample image set by summation; determining a weighted loss by weighted summation of the global feature similarity total loss, the key point position deviation total loss and the classification total loss; determining whether the weighted loss converges or is less than a predetermined threshold; and in the case of a negative determination, optimizing the face recognition model by adjusting parameters of the face recognition model; wherein the unoccluded image and the occluded image included in each face sample image pair are an unoccluded face sample image and an occluded face sample image of the same person; each fusion feature in the fusion feature pair of the face sample image pair is a feature obtained by fusing global features and local features of the corresponding face sample image; and the local features of the corresponding face sample image are associated with unoccluded face local regions.

[0007] According to an aspect of the present disclosure, an apparatus for training a face recognition model is provided. The apparatus comprises a memory having instructions stored thereon; and one or more processors capable of communicating with the memory to execute the instructions retrieved from the memory, and the instructions cause the one or more processors to: for each face sample image pair comprising an unoccluded image and an occluded image in a set of face sample images: determine a global feature similarity loss of the face sample image pair based on a pair of global features of the face sample image pair; determine a key point position deviation loss of the face sample image pair based on a pair of N key point positions of the face sample image pair; and determine a classification loss of the face sample image pair based on a pair of fusion features of the face sample image pair; determine a global feature similarity total loss, a key point position deviation total loss and a classification total loss for the set of face sample images by summation; determine a weighted loss by weighted summation of the global feature similarity total loss, the key point position deviation total loss and the classification total loss; determine whether the weighted loss converges or is less than a predetermined threshold; and in a case where the determination result is "no", optimize the face recognition model by adjusting parameters of the face recognition model; wherein the unoccluded image and the occluded image comprised in each face sample image pair are an unoccluded face sample image and an occluded face sample image of the same person; each fusion feature in the pair of fusion features of the face sample image pair is a feature obtained by fusing global features and local features of the respective face sample images; and the local features of the respective face sample images are associated with unoccluded face local regions.

[0008] According to an aspect of the present disclosure, a computer readable storage medium having a program stored thereon is provided. The program causes a computer running the program to: for each face sample image pair including an unoccluded image and an occluded image in a face sample image set: determine a global feature similarity loss of the face sample image pair based on a global feature pair of the face sample image pair; determine a key point position deviation loss of the face sample image pair based on an N key point position pair of the face sample image pair; and determine a classification loss of the face sample image pair based on a fusion feature pair of the face sample image pair; determine a global feature similarity total loss, a key point position deviation total loss and a classification total loss for the face sample image set by summation; determine a weighted loss by weighted summation of the global feature similarity total loss, the key point position deviation total loss and the classification total loss; determine whether the weighted loss converges or is less than a predetermined threshold; and in a case where the determination result is "no", optimize a face recognition model by adjusting parameters of the face recognition model; wherein the unoccluded image and the occluded image included in each face sample image pair are an unoccluded face sample image and an occluded face sample image of the same person; each fusion feature in the fusion feature pair of the face sample image pair is a feature obtained by fusing a global feature and a local feature of a corresponding face sample image; and the local feature of the corresponding face sample image is associated with an unoccluded face local region.

[0009] The method, device and storage medium of the present disclosure have at least the following beneficial effects: improving the face recognition accuracy for occluded face images. BRIEF DESCRIPTION OF DRAWINGS

[0010] Embodiments of the present disclosure are described below with reference to the accompanying drawings, which are intended to facilitate a more readily understood explanation of the above-mentioned and other purposes, features and advantages of the present disclosure. The accompanying drawings are merely intended to illustrate the principles of the present disclosure. The size and relative positions of elements in the drawings are not necessarily drawn to scale. The same reference numbers can be used to indicate the same elements. In the drawings:

[0011] Figure 1 An exemplary flowchart of a method for training a face recognition model according to one embodiment of the present disclosure is shown;

[0012] Figure 2 An exemplary flowchart of a method for determining a first local feature according to one embodiment of the present disclosure is shown;

[0013] Figure 3 An exemplary flowchart of a method for face recognition according to one embodiment of the present disclosure is shown;

[0014] Figure 4 An apparatus for training a face recognition model according to one embodiment of the present disclosure is shown;

[0015] Figure 5 An apparatus for training a face recognition model is shown in accordance with one embodiment of the present disclosure; and

[0016] Figure 6 An exemplary block diagram of an information processing device is shown in accordance with one embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] In the following, exemplary embodiments of the present disclosure will be described with reference to the drawings. In the description, not all of the features of a practical embodiment are described in order to make the description clear and concise. It should be appreciated, however, that many embodiment-specific decisions can be made in the course of developing any such practical embodiment in order to achieve the specific objectives of the developer, and these decisions can vary from embodiment to embodiment.

[0018] It is also to be noted that, in order to avoid obscuring the present disclosure due to unnecessary details, only the structures of the apparatuses closely related to the solutions according to the present disclosure are shown in the drawings, and other details not closely related to the present disclosure are omitted.

[0019] It should be understood that the present disclosure is not limited to the described embodiments only by virtue of the following description in relation to the drawings. In this text, the embodiments can be combined with each other, features replaced or borrowed between different embodiments, and one or more features omitted in one embodiment, where possible.

[0020] Computer program code for carrying out operations of aspects of embodiments of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages.

[0021] The methods of the present disclosure can be implemented by a circuit having a corresponding functional configuration. The circuit includes a circuit for a processor.

[0022] The inventors have studied the recognition of face images with occlusion. For the recognition of face images with occlusion, if the global features extracted from the whole face are still used for face recognition as in the conventional face recognition model, the recognition accuracy will decrease due to the existence of the occlusion. If the occluded part is discarded and only the image without occlusion is used to extract features, the benefits brought by the global features to the recognition are also discarded, and the performance of the recognition model can also be degraded. Considering the foregoing, the inventors propose a scheme of performing face recognition by using the fusion features of face images, wherein the fusion features are features obtained by fusing the global features and the local features of the face images, and the local features are associated with n key point positions. In training the face recognition model, in addition to optimizing the model by using the classification loss, the feature extraction is also constrained by using the global feature similarity and the key point position deviation, so that the final obtained fusion features contain more useful information to improve the accuracy of face recognition. The n key points corresponding to the n key point positions are selected from the key points in the non-occluded face local region. The non-occluded face local region is related to the cover that causes the occlusion. For example, when the cover is a mask, the non-occluded face local region can be selected as the face local region above and including the eyes. Therefore, when the cover is a mask, the non-occluded face local region can be defined as the face local region above and including the eyes.

[0023] One aspect of the present disclosure provides a method for training a face recognition model. The method is described below with reference to Figure 1 .

[0024] Figure 1 An exemplary flowchart of a method 100 for training a face recognition model Mfr is shown according to one embodiment of the present disclosure.

[0025] At step S101, for each face sample image pair imP[i] in the face sample image set SimP including an unoccluded image imP[i].Im1 (which can be simply written as Im1[i] or Im1) and an occluded image imP[i].Im2 (which can be simply written as Im2[i] or Im2): determining a global feature similarity loss lossG[i] (which is simply written as lossG in the figure) of the face sample image pair based on a global feature pair (imP[i].Im1.Fg, imP[i].Im2.Fg) of the face sample image pair; determining a key point position deviation loss lossP[i] (which is simply written as lossP in the figure) of the face sample image pair based on an N key point position pair of the face sample image pair; and determining a classification loss lossC[i] (which is simply written as lossC in the figure) of the face sample image pair based on a fusion feature pair (imP[i].Im1.Ff, imP[i].Im2.Ff) of the face sample image pair. For example, i is a natural number in the range of [1, i_max]. The global feature pair (imP[i].Im1.Fg, imP[i].Im2.Fg) can be simply written as (Fg1[i], Fg2[i]) or (Fg1, Fg2), Fg1[i] is the global feature of the unoccluded face sample image Im1[i], also referred to as the “first global feature” herein, and Fg2[i] is the global feature of the occluded face sample image Im2[i], also referred to as the “second global feature” herein. The fusion feature pair (imP[i].Im1.Ff, imP[i].Im2.Ff) can be simply written as (Ff1[i], Ff2[i]) or (Ff1, Ff2), Ff1[i] is the fusion feature of the unoccluded face sample image Im1[i], simply referred to as the “first fusion feature”, and Ff2[i] is the fusion feature of the occluded face sample image Im2[i], simply referred to as the “second fusion feature”. The unoccluded image Im1[i] and the occluded image Im2[i] included in each face sample image pair imP[i] are an unoccluded face sample image and an occluded face sample image of the same person. Each fusion feature in the fusion feature pair (Ff1[i], Ff2[i]) of the face sample image pair imP[i] is a feature obtained by fusing the global feature and the local feature of the corresponding face sample image: the first fusion feature Ff1[i] is a feature obtained by fusing the first global feature Fg1[i] and the first local feature Fl1[i], and the second fusion feature Ff2[i] is a feature obtained by fusing the second global feature Fg2[i] and the second local feature Fl2[i], Fl1[i] represents the local feature of the unoccluded face sample image Im1[i], simply referred to as the “first local feature”, and Fl2[i] represents the local feature of the occluded face sample image Im2[i], simply referred to as the “second local feature”. The first and second local features are associated with the unoccluded face local region.The unoccluded face local region is related to the cover that causes the occlusion. Therefore, when the cover is given, it can be considered that, for both the occluded image and the unoccluded image, the face region therein exists a face local region corresponding to the unoccluded face local region determined by the given cover, and the local feature can be determined by using the face local region. N is a natural number greater than 1.

[0026] At step S103, the global feature similarity total loss, the key point position deviation total loss LossP and the classification total loss LossC for the face sample image set SimP are determined by summation. Each total loss can be calculated according to the following formula:

[0027]

[0028]

[0029]

[0030] At step S105, the weighted loss Lw is determined by weighted summation of the global feature similarity total loss LossG, the key point position deviation total loss LossP and the classification total loss LossC (see the following formula, preferably w3> w2 and w3> w1):

[0031] Lw = w1*LossG + w2*LossP + w3*LossC.

[0032] At step S107, it is determined whether the weighted loss Lw converges or is less than a predetermined threshold. When the determination result is "yes", the method ends, and the model training ends; when the determination result is "no", step S109 is executed. It is preferred to judge whether to end the training based on the convergence condition. For example, when the difference of Lw of adjacent two times of iteration training is less than a preset threshold, it is considered that Lw has converged, and the training is ended.

[0033] At step S109, the face recognition model Mfr is optimized by adjusting the parameters of the face recognition model Mfr, that is, the face recognition model Mfr is updated. After completing step S109, return to step S101 to iteratively train the model Mfr until the weighted loss Lw converges or is less than a predetermined threshold. The parameters of the face recognition model Mfr can be adjusted based on Lw using the gradient descent method to optimize the face recognition model Mfr.

[0034] Further details of the method for training the face recognition model of the present disclosure are described below.

[0035] In an embodiment, the N key point positions of each face sample image in the pair of face sample images correspond to N face key points in a predetermined face key point set Sp1. The local feature of the target face sample image is associated with n key point positions of the N key point positions of the target face sample image. The n key point positions of the target face sample image correspond to n face key points in a predetermined subset Sp2 of the predetermined face key point set Sp1. The n face key points are face key points in an unoccluded face local region. The key points included in the predetermined face key point set Sp1 and the predetermined subset Sp2 are key points in a general sense, and are irrelevant to which person the face corresponds to. For example, for the pair of face sample images imP[i], the N face key points corresponding to the N face key point positions of the unoccluded image Im1[i] are the key points indicated by the N face key points in the predetermined face key point set Sp1, the first local feature Fl1[1] of the unoccluded image Im1[i] is associated with n key point positions of the aforementioned N key point positions of Im1[i], and the n face key points are face key points in an unoccluded face local region and correspond to the n face key points in the predetermined subset Sp2, that is, the n key points associated with Fl1[i] are selected from the aforementioned N key points in the unoccluded face local region (for example, N = 103, n = 3); the N face key points corresponding to the N face key point positions of the occluded image Im2[i] are the key points indicated by the N face key points in the predetermined face key point set Sp1, the second local feature Fl2[1] of Im2[i] is associated with n key point positions of the aforementioned N key point positions of Im2[i], and the n face key points are face key points in an unoccluded face local region, that is, the n key points associated with Fl2[i] are selected from the aforementioned N key points in the unoccluded face local region of Im2[i].

[0036] The occluded face sample image is a face sample image in which a local region of the face is covered by a cover relative to the unoccluded face sample image. The occluded face image is a face image in which a local region of the face is covered by a cover. In the present disclosure, the cover includes but is not limited to a mask, a scarf, a face mask, sunglasses, a bandage, a hat, or an eye patch. It is considered that the region covered by different covers is relatively fixed, and it is preferred to train a corresponding face recognition model for each cover (for example, a mask).

[0037] In an embodiment, the pair of face sample images for a person can be obtained by respectively shooting the unoccluded face image and the occluded face image of the person with and without a cover on the face of the person.

[0038] In one embodiment, the occluded face sample image Im2[i] is obtained by using an image processing module to add an overlay covering part of the face to the face in the unoccluded face sample image Im1[i]:

[0039] Im2[i] = Addm(Im1[i]),

[0040] where Addm is an image processing module that provides an occluded face image by adding an overlay covering part of the face to the face, referred to as an “overlay adding module”. It can be appreciated that obtaining occluded face sample images in this way has the following benefits: in the field of face recognition, there are already a large number of labeled unoccluded face sample images, therefore, the occluded face sample images obtained by the image processing module Addm can directly use the labels of the corresponding unoccluded face sample images, without the need for manual labeling or other ways to obtain labels, so that a large number of occluded face sample images required for training a face recognition model can be obtained at low cost and accurately.

[0041] In one embodiment, global features are extracted from face images using a Siamese network comprising two networks Siamese1 and Siamese2, which can be represented as:

[0042] Fg1[i] = Siamese1(Im1[i]);

[0043] Fg2[i] = Siamese2(Im2[i]).

[0044] The global feature extraction networks Siamese1 and Siamese2 are isomorphic and share weight parameters. The Siamese network can use two task branches with CNN (convolutional neural network) layers to respectively generate two feature maps (i.e., two global features) from two input images Im1[i], Im1[2] respectively. The Siamese network can be composed of any CNN network. The global feature pair of the face sample image pair imP[i] includes the first global feature Fg1[i] and the second global feature Fg2[i] extracted from the unoccluded image Im1[i] and the occluded image Im2[i] respectively using the Siamese network. The Siamese network is part of the face recognition model of the present disclosure, and during the training of the face recognition model, the network parameters of the Siamese network will be updated and optimized. During the training process, adjusting the parameters of the face recognition model includes adjusting the parameters of the Siamese network

[0045] In an embodiment, the global feature similarity loss lossG[i] of the face sample image pair imP[i] can be determined in the following manner: the global feature similarity loss lossG[i] is determined based on the similarity of the first global feature Fg1[i] and the second global feature Fg2[i]. The global feature similarity loss lossG[i] of the face sample image pair imP[i] is associated with the similarity sim[i] of the first global feature Fg1[i] and the second global feature Fg2[i]. The greater the similarity sim[i], the smaller the global feature similarity loss lossG[i]. For example, the lossG[i] is positively correlated with "1-sim[i]".

[0046] To ensure the quality of the extracted global features, the extraction of the global features can be optimized using features related to the global features. In an embodiment, the extraction of the global features can be optimized using pose features related to the global features. For example, the weighted loss Lw further includes a total pose similarity loss LossZ for the face sample image set. The total pose similarity loss LossZ for the face sample image set is the sum of the pose feature similarity loss lossZ of the pose feature pair (Fz1[i], Fz2[i]) of each face sample image pair imP[i] in the face sample image set:

[0047]

[0048] The pose feature pair includes the first pose feature Fz1[i] of the corresponding unoccluded image Im1[i] in the corresponding face sample image pair imP[i] and the second pose feature Fz2[i] of the corresponding occluded image Im2[i] in the face sample image pair imP[i]. The first pose feature Fz1[i] is a feature representing the head pose of the corresponding unoccluded image determined based on the global feature Fg1[i] of the corresponding unoccluded image. The second pose feature Fz2[i] is a feature representing the head pose of the corresponding occluded image determined based on the global feature Fz2[i] of the corresponding occluded image. The pose feature similarity loss lossZ[i] of the face sample image pair imP[i] is associated with the similarity simz[i] of the first pose feature Fz1[i] and the second pose feature Fz2[i]. The greater the similarity simz[i], the smaller the pose feature similarity loss lossZ[i]. For example, the lossZ[i] is positively correlated with "1-simz[i]". The pose feature can be extracted using a pose extraction sub-model Mpose:

[0049] Fz1[i] = Mpose(Fg1[i]);

[0050] Fz2[i] = Mpose(Fg2[i]).

[0051] The parameters of the pose extraction sub-model Mpose can be adjusted and optimized during the Mfr training process.

[0052] In an embodiment, when determining the key point position deviation loss, the N key points of each face sample image in the face sample image pair correspond to the N face key points in the predetermined face key point set Sp1. The key points in the face image have position attributes, so different key points can be distinguished by position. Each key point position can be represented using coordinates. For a face image, multiple points can be selected as face key points as the basis for face recognition. Therefore, for a specific face image, N face key points can be obtained, for example, by detection, and the N face key points can be part or all of the aforementioned multiple face key points. For each specific face image, the key points corresponding to the N face key points in the predetermined face key point set Sp1 can be determined. The predetermined face key point set Sp1 consisting of N face key points can be determined in advance when configuring the face recognition model. The key point position deviation loss is related to the N face key points in the predetermined face key point set Sp1. By determining the N key points P1[i][1] to P1[i][N] of the unoccluded face image Im1[i] in the face sample image pair imP[i], and determining the N key points P2[i][1] to P2[i][N] of the occluded face image Im2[i] in the face sample image pair imP[i], the N key point position pairs (P1[i][1], P2[i][1]) to ((P1[i][N], P2[i][N]) of the face sample image pair imP[i] are determined, wherein each key point position pair corresponds to the position pair of the same face key point in Im1[i] and Im2[2] (for example, the key point position pair consisting of the position coordinates of the left corner of the eye in Im1 and the position coordinates of the left corner of the eye in Im2 corresponds to the same face key point, the left corner of the face eye). Each key point position pair can be represented as: (P1[i][k], P2[i][k]), k is one of 1 to N; P1[i][k] represents the position of the kth key point in Im1[i], P2[i][k] represents the position of the kth key point in Im2[i], the kth key point in Im1[i] and the kth key point in Im2[i] both correspond to the kth face key point in the predetermined face key point set Sp1, for example, both correspond to the kth face key point in Sp1, the left corner of the face eye (i.e., the kth key point in Im1[i] and the kth key point in Im2[i] are both, for example, the left corner of the face eye).The N key point positions P1[i][1] to P1[i][N] of the unoccluded image Im1[i] can be determined using the key point detection sub-model Mde, denoted as: {P1[i][k]} = Mde(Im1[i]), where {P1[i][k]} is the position set of the N key points of Im1[i]; the N key points P2[i][1] to P2[i][N] of the occluded image Im2[i] can be determined based on the second global feature Fg2[i] using the key point prediction sub-model Mpp, denoted as: {P2[i][k]} = Mpp(Fg2[i]), where {P2[i][k]} is the position set of the N key points of Im2[i]. The key point detection sub-model Mde can be a pre-configured model whose parameters are not changed during the training of Mfr. Mde can detect N key points corresponding to the N facial key points in the predetermined facial key point set Sp1 from the unoccluded image and give their positions; Mpp can predict N key points of the occluded image corresponding to the N facial key points in the predetermined facial key point set Sp1 based on the second global feature and give their positions. The facial sample image pair imP[i] is associated with the key point position deviation loss lossP[i] and the deviation between the two key points in the key point position pair corresponding to the same facial key point in the N key point position pairs, for example, lossP[i] is proportional to the sum SumD of the distances dis between the N key point pairs (i.e., the distance between the two key points corresponding to the same facial key point in Sp1 in the key point pair).

[0053]

[0054] where dis[i][k] represents the distance between the kth key point in Im1[i] and the kth key point in Im[2]. When the cover is a mask, the n facial key points constituting the predetermined subset Sp2 are preferably key points related to eyes, eyebrows, and forehead. The parameters of the key point prediction sub-model Mpp can be determined during the training of Mfr.

[0055] In one embodiment, the local features are determined based on n key points associated with the unoccluded facial local region and global features, and the n key points are part of the N key points related to the key point position deviation loss. Specifically, the first local feature Fl1[i] of the unoccluded image Im1[i] is determined based on the n key point positions of the unoccluded image Im1[i] and the first global feature Fg1[i]; the second local feature Fl2[i] of the occluded image Im2[i] is determined based on the n key points of the occluded image Im2[i] and the second global feature Fg2[i].

[0056] Figure 2An exemplary flowchart of a method 200 of determining the first local feature is shown according to one embodiment of the present disclosure. The first local feature Fl1[i] of the unoccluded image Im1[i] is determined in the method 200 by: at step S201, generating an intermediate feature Fm1[i] by extracting a plurality of components from the first global feature Fg1[i] using a window mask that determines n windows of the unoccluded image Im1[i] by the n keypoint positions (P1[i][1] to P1[i][n]); at step S203, determining a weight matrix W1c[i] by convolution processing on the intermediate feature Fm1[i]; and at step S205, determining the product of the first global feature Fg1[i] and the weight matrix W1c[i] as the first local feature Fl1[i]. Wherein, in the case that the intermediate feature Fm1[i] is aligned with the window mask, the components of the intermediate feature Fm1[i] that are within each window of the window mask are equal to the components at the same positions of the first global feature Fg1[i]; and the components of the intermediate feature Fm1[i] that are outside the n windows of the window mask are equal to zero or a predetermined small constant. This can be expressed in the following equations:

[0057] Fm1[i] = Win(Fg1[i]);

[0058] W1c[i] = Conv(Fm1[i]);

[0059] Fl1[i] = Fg1[i] * W1c[i];

[0060] Wherein, Win represents the operation of extracting components from a feature using a window mask to determine an intermediate feature; and Conv represents convolution processing. For example, when the number of n keypoint positions is 3, the window mask has 3 windows that are centered at the 3 keypoint positions respectively. The size of the window is, for example, 11*11. When the number of n keypoint position pairs is n, the number of windows of the window mask is also n, and the sizes of the n windows can be different from each other. In other words, the (first or second) intermediate feature is determined by extracting partial components of the global feature based on the n keypoint positions, where the positions of the partial components are determined by the n keypoint positions, and the local feature is determined based on the intermediate feature.

[0061] In a similar way, the second local feature F2l[i] of the occluded image Im2[i] is determined by the present embodiment:

[0062] Fm2[i] = Win(Fg2[i]);

[0063] W2c[i] = Conv(Fm2[i]);

[0064] Fl2[i] = Fg1[i] * W2c[i].

[0065] The parameters of the convolution process Conv are gradually optimized and determined during the training process. Adjusting the parameters of the face recognition model during the training process includes adjusting the parameters of the convolution process Conv.

[0066] In one embodiment, the first global feature Fg1[i] and the first local feature Fl1[i] are spliced together as the first fusion feature Ff1[i]; and the second global feature Fg2[i] and the second local feature Fl2[i] are spliced together as the second fusion feature Ff2[i].

[0067] In one embodiment, the sum of the product of the first global feature Fg1[i] and the first local feature Fl1[i] and the fusion weight a is taken as the first fusion feature Ff1[i]; and the sum of the product of the second global feature Fg2[i] and the second local feature Fl2[i] and the fusion weight a is taken as the second fusion feature Ff2[i]. This can be expressed as the following formula:

[0068] Ff1[i] = Fg1[i] + a * Fl1[i];

[0069] Ff2[i] = Fg2[i] + a * Fl2[i].

[0070] Wherein, as the parameter of Mfr, the fusion weight a is adjusted and gradually determined during the training of Mfr. Adjusting the parameters of the face recognition model during the training process includes adjusting the fusion weight a.

[0071] In one embodiment, the classification total loss is a softmax loss. The classification loss lossC[i] of the face sample image pair imP[i] can be the sum of the classification loss of the unoccluded image and the classification loss of the occluded image. The face classification model Mfr includes a classification sub-model Mc. The classification sub-model Mc is a neural network model, which is used to classify the face image based on the fusion feature. For example, the classification sub-model Mc classifies (i.e., recognizes) the unoccluded image Im1[i] based on the first fusion feature Fl1[i], thereby obtaining the first classification loss for the unoccluded image Im1[i]; and the classification sub-model Mc classifies (i.e., recognizes) the occluded image Im2[i] based on the second fusion feature Fl2[i], thereby obtaining the second classification loss for the occluded image Im2[i]. The sum of the first and second classification losses can be taken as the classification loss lossC[i] of the face sample image pair imP[i]. Adjusting the parameters of the face recognition model during the training process includes adjusting the model parameters of the classification sub-model Mc.

[0072] The method for training a face recognition model of the present disclosure comprises using multiple losses to improve the acquisition of features for classification, thereby obtaining fused features, and improving the recognition accuracy of recognizing face images with occlusions. The model training method of the present disclosure is performed based on image pairs (unoccluded image, occluded image), and through feature similarity constraints and key point position deviation loss constraints, the similarity of the two global features in the global feature pair determined by the trained face recognition model can be improved. The model training method of the present disclosure enhances the resolution capability of the local features extracted in the face with occlusions by using face key points to generate local feature extraction regions and weighting and enhancing the local features.

[0073] One aspect of the present disclosure provides a face recognition method. Reference is made below to Figure 3 The face recognition method is exemplarily described. Figure 3 An exemplary flowchart of a method 300 of face recognition according to one embodiment of the present disclosure is shown. At step S301, a face recognition model Mfr is trained using the method for training a face recognition model of the present disclosure. At step S303, a face recognition is performed on a face image to be recognized using the trained face recognition model. The face image to be recognized can be an unoccluded face image, or can be an occluded face image. The face recognition model Mfr trained using the method for training a face recognition model of the present disclosure is particularly suitable for face recognition of occluded face images. Step S303 can comprise determining a fused feature of the face image to be recognized, which is determined by fusing a global feature and a local feature of the face image to be recognized. Step S303 can further comprise classifying (i.e., recognizing) a face in the face image to be recognized based on the fused feature of the face image to be recognized. The method 300 can comprise determining whether there is a covering on the face in the face image to be recognized. When the determination result is “yes”, N key points in the face image to be recognized are determined using a key point predictor model; and when the determination result is “no”, N key points in the face image to be recognized are determined using a key detector model. The method 300 corresponds to the method 100 for training a face recognition model, and further details of the method 300 can be referred to the description of the method 100.

[0074] One aspect of the present disclosure provides an apparatus for training a face recognition model. Reference is made below to Figure 4 The apparatus is described. Figure 4An apparatus 400 for training a face recognition model is shown according to one embodiment of the present disclosure. The apparatus 400 includes a weighted loss determining unit 401, a judging unit 403 and an optimizing unit 405. The weighted loss determining unit 401 is configured to: for each face sample image pair including an unoccluded image and an occluded image in a face sample image set: determine a global feature similarity loss lossG of the face sample image pair based on a global feature pair of the face sample image pair; determine a key point position deviation loss lossP of the face sample image pair based on an N key point position pair of the face sample image pair; and determine a classification loss lossC of the face sample image pair based on a fusion feature pair of the face sample image pair; determine a global feature similarity total loss LossG, a key point position deviation total loss LossP and a classification total loss LossC for the face sample image set by summation; determine a weighted loss Lw by weighted summation of the global feature similarity total loss, the key point position deviation total loss and the classification total loss. The judging unit 403 is configured to: determine whether the weighted loss Lw converges or is less than a predetermined threshold. The optimizing unit 405 is configured to: in a case where the determination result is "no", optimize the face recognition model Mfr by adjusting parameters of the face recognition model. The unoccluded image and the occluded image included in each face sample image pair are an unoccluded face sample image and an occluded face sample image of a same person. The unoccluded image and the occluded image included in each face sample image pair are an unoccluded face sample image and an occluded face sample image of a same person; each fusion feature in the fusion feature pair of the face sample image pair is a feature obtained by fusing global features and local features of the corresponding face sample image; and the local features of the corresponding face sample image are associated with unoccluded face local regions. The apparatus 400 has a corresponding relationship with the method 100, and further details of the apparatus 400 can be referred to the description of the method 100 herein.

[0075] According to one aspect of the present disclosure, an apparatus for training a face recognition model is provided. The following refers to the drawings, in which Figure 5 The apparatus is described. Figure 5An apparatus 500 for training a face recognition model is shown according to one embodiment of the present disclosure. The apparatus comprises a memory 501 having instructions stored thereon, and one or more processors 503 that are capable of communicating with the memory to execute the instructions retrieved from the memory, and the instructions cause the one or more processors to: for each face sample image pair comprising an unoccluded image and an occluded image in a set of face sample images: determine a global feature similarity loss for the face sample image pair based on a pair of global features of the face sample image pair; determine a key point position deviation loss for the face sample image pair based on a pair of N key point positions of the face sample image pair; and determine a classification loss for the face sample image pair based on a pair of fusion features of the face sample image pair; determine a global feature similarity total loss, a key point position deviation total loss, and a classification total loss for the set of face sample images by summing; determine a weighted loss by weighted summing the global feature similarity total loss, the key point position deviation total loss, and the classification total loss; determine whether the weighted loss converges or is less than a predetermined threshold; and in a case where the determination result is "no", optimize the face recognition model by adjusting parameters of the face recognition model; wherein each fusion feature in the pair of fusion features of the face sample image pair is a feature obtained by fusing a global feature and a local feature of a respective face sample image; and the local feature of the respective face sample image is associated with an unoccluded face local region. The apparatus 500 has a corresponding relationship with the method 100, and further details of the apparatus 500 can be referred to the description of the method 100 herein.

[0076] One aspect of the present disclosure provides a computer readable storage medium having stored thereon a program. The program causes a computer on which the program runs to: for each face sample image pair including an unoccluded image and an occluded image in a face sample image set: determine a global feature similarity loss of the face sample image pair based on a global feature pair of the face sample image pair; determine a key point position deviation loss of the face sample image pair based on an N key point position pair of the face sample image pair; and determine a classification loss of the face sample image pair based on a fusion feature pair of the face sample image pair; determine a global feature similarity total loss, a key point position deviation total loss and a classification total loss for the face sample image set by summation; determine a weighted loss by weighted summation of the global feature similarity total loss, the key point position deviation total loss and the classification total loss; determine whether the weighted loss converges or is less than a predetermined threshold; and in a case where the determination result is "no", optimize a face recognition model by adjusting parameters of the face recognition model; wherein each fusion feature in the fusion feature pair of the face sample image pair is a feature obtained by fusing a global feature and a local feature of a corresponding face sample image; and the local feature of the corresponding face sample image is associated with an unoccluded face local region. The program corresponds to the method 100, and further details of the program can be referred to the description of the method 100 herein.

[0077] According to one aspect of the present disclosure, an information processing apparatus is also provided.

[0078] Figure 6 is an exemplary block diagram of an information processing apparatus 600 according to one embodiment of the present disclosure. In Figure 6 A central processing unit (CPU) 601 performs various processing according to a program stored in a read only memory (ROM) 602 or a program loaded from a storage section 608 to a random access memory (RAM) 603. In the RAM 603, data and the like required when the CPU 601 performs various processing are also stored as necessary.

[0079] The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output interface 605 is also connected to the bus 604.

[0080] The following components are connected to the input / output interface 605: an input section 606 including a keyboard and the like; an output section 607 including a display such as a liquid crystal display (LCD) and a speaker and the like; a storage section 608 such as a hard disk; and a communication section 609 including a network interface card such as a LAN card, a modem, and the like. The communication section 609 performs communication processing via a network such as the Internet, a local area network, a mobile network, or a combination thereof.

[0081] The drive 66 is also connected to the input / output interface 605 as necessary. A removable medium 611 such as a semiconductor memory or the like is attached to the drive 610 as necessary, so that a program read therefrom is installed in the storage section 608 as necessary.

[0082] The CPU 601 can execute a program for implementing the method for training a face recognition model according to the present disclosure or a program for recognizing a face.

[0083] While the present application has been disclosed by the description of specific embodiments thereof, it is understood that various modifications (including combinations or substitutions of features among the embodiments, or replacements, if appropriate, within the scope of the application) can be made by those skilled in the art in its spirit and scope as defined by the appended claims. Such modifications, improvements or equivalents are also to be construed to fall within the scope of the present application.

[0084] It should be emphasized that the term "comprises / comprising" when used in this specification is taken to specify the presence of stated features, elements, steps or components but does not preclude the presence or addition of one or more other features, elements, steps, components or groups thereof.

[0085] Furthermore, the methods of the embodiments of the present application are not limited to being executed in the time sequence described in the specification or shown in the drawings, but can be executed in other time sequences, in parallel or independently. Therefore, the execution sequence of the methods described in the specification does not limit the technical scope of the present application.

[0086] Note

[0087] The present disclosure includes, but is not limited to, the following technical solutions.

[0088] 1. A method for training a face recognition model, comprising:

[0089] For each face sample image pair including an unoccluded image and an occluded image in a face sample image set:

[0090] determining a global feature similarity loss of the face sample image pair based on the global feature pair of the face sample image pair;

[0091] determining a key point position deviation loss of the face sample image pair based on the N key point position pair of the face sample image pair; and

[0092] determining a classification loss of the face sample image pair based on the fusion feature pair of the face sample image pair;

[0093] determining a global feature similarity total loss, a key point position deviation total loss and a classification total loss for the face sample image set by summation;

[0094] determine a weighted loss by weighted sum of the global feature similarity total loss, the key point position deviation total loss, and the classification total loss;

[0095] determine whether the weighted loss converges or is less than a predetermined threshold; and

[0096] in a case where the determination result is “No”, optimize the face recognition model by adjusting parameters of the face recognition model;

[0097] wherein,

[0098] the unoccluded image and the occluded image included in each face sample image pair are an unoccluded face sample image and an occluded face sample image of the same person;

[0099] each fusion feature in the fusion feature pair of the face sample image pair is a feature obtained by fusing global features and local features of the corresponding face sample image; and

[0100] the local features of the corresponding face sample image are associated with unoccluded face local regions.

[0101] 2. The method of appendage 1, wherein the occluded face sample image is obtained by using an image processing module to add a cover covering part of the face to the face in the unoccluded face sample image.

[0102] 3. The method of appendage 1, wherein the occluded image is a face image with a local region covered by a cover; and

[0103] the cover is a mask, a scarf, a hat, an eye patch, or sunglasses.

[0104] 4. The method of appendage 1, wherein the global feature pair of the face sample image pair includes a first global feature and a second global feature extracted from the unoccluded image and the occluded image respectively using a twin network.

[0105] 5. The method of appendage 1, wherein the weighted loss further includes a pose similarity total loss for the face sample image set:

[0106] the pose similarity total loss of the face sample image set is a sum of pose feature similarity losses of pose feature pairs of each face sample image pair in the face sample image set;

[0107] the pose feature pair includes a first pose feature of a corresponding unoccluded image in the corresponding face sample image pair and a second pose feature of a corresponding occluded image in the corresponding face sample image pair;

[0108] the first pose feature is a feature representing a head pose of the respective unoccluded image determined based on global features of the respective unoccluded image; and

[0109] the second pose feature is a feature representing a head pose of the respective occluded image determined based on global features of the respective occluded image.

[0110] 6. The method according to append 1, wherein each keypoint position pair in the N keypoint position pairs of the face sample image pair comprises:

[0111] a first keypoint position: a position of an Nth keypoint corresponding to an Nth face keypoint in the predetermined face keypoint set detected by the keypoint detection sub-model from the unoccluded image; and

[0112] a second keypoint position: a position of an Nth keypoint corresponding to an Nth face keypoint in the predetermined face keypoint set predicted by the keypoint prediction sub-model based on the global feature of the occluded image in the global feature pair of the face sample image pair.

[0113] 7. The method according to append 1, wherein the N keypoint positions of each face sample image in the face sample image pair correspond to N face keypoints in the predetermined face keypoint set;

[0114] the local feature of the respective face sample image is associated with n keypoint positions in the N keypoint positions of the respective face sample image;

[0115] the n keypoint positions of the respective face sample image correspond to n face keypoints in a predetermined subset of the predetermined face keypoint set; and

[0116] the n face keypoints are face keypoints within the unoccluded face local region.

[0117] 8. The method according to append 7, wherein the fused feature pair of the face sample image pair comprises a first fused feature and a second fused feature;

[0118] the first fused feature is determined by fusing the first global feature of the unoccluded image in the global feature pair of the face sample image pair and the first local feature of the unoccluded image; and

[0119] the second fused feature is determined by fusing the second global feature of the occluded image in the global feature pair of the face sample image pair and the second local feature of the occluded image.

[0120] 9. The method according to append 8, wherein the first local feature of the unoccluded image is determined by:

[0121] generating intermediate features by extracting a plurality of components from a first global feature of the unoccluded image using a window mask having n windows determined by n key point positions of the unoccluded image;

[0122] determining a weight matrix by performing convolution processing on the intermediate features; and

[0123] determining a product of the first global feature and the weight matrix as the first local feature;

[0124] wherein, in a case where the intermediate features are aligned with the window mask, components of the intermediate features located within each window of the window mask are equal to components at the same positions of the first global feature; and components of the intermediate features located outside the n windows of the window mask are equal to zero or a predetermined small constant.

[0125] 10. The method of claim 8, wherein the first local feature of the unoccluded image is determined based on n key point positions of the unoccluded image and the first global feature; and

[0126] the second local feature of the occluded image is determined based on n key points of the occluded image and the second global feature.

[0127] 11. The method of claim 8, wherein the first global feature and the first local feature are concatenated together as the first fused feature; and

[0128] the second global feature and the second local feature are concatenated together as the second fused feature.

[0129] 12. The method of claim 8, wherein a sum of a product of the first global feature and the first local feature and a fusion weight is taken as the first fused feature; and

[0130] a sum of a product of the second global feature and the second local feature and the fusion weight is taken as the second fused feature.

[0131] 13. The method of claim 1, wherein the classification total loss is a softmax loss.

[0132] 14. An apparatus for training a face recognition model, comprising:

[0133] a memory having instructions stored thereon; and

[0134] one or more processors capable of communicating with the memory to execute instructions retrieved from the memory and the instructions cause the one or more processors to:

[0135] for each face sample image pair including an unoccluded image and an occluded image in a face sample image set:

[0136] determine a global feature similarity loss for the face sample image pair based on a pair of global features of the face sample image pair;

[0137] determine a key point position deviation loss for the face sample image pair based on a pair of N key point positions of the face sample image pair; and

[0138] determine a classification loss for the face sample image pair based on a pair of fusion features of the face sample image pair;

[0139] determine a global feature similarity total loss, a key point position deviation total loss and a classification total loss for the face sample image set by summation;

[0140] determine a weighted loss by weighted summation of the global feature similarity total loss, the key point position deviation total loss and the classification total loss;

[0141] determine whether the weighted loss converges or is less than a predetermined threshold; and

[0142] in a case where the determination result is "no", optimize the face recognition model by adjusting parameters of the face recognition model;

[0143] wherein the unoccluded image and the occluded image included in each face sample image pair are an unoccluded face sample image and an occluded face sample image for the same person;

[0144] each fusion feature in the pair of fusion features of the face sample image pair is a feature obtained by fusing a global feature and a local feature of a corresponding face sample image; and

[0145] the local feature of the corresponding face sample image is associated with an unoccluded face local region.

[0146] 15. The apparatus according to Appendix 13, wherein the occluded face sample image is an image obtained by adding a cover covering part of a face in the unoccluded face sample image to the face.

[0147] 16. The apparatus according to Appendix 14, wherein,

[0148] N key point positions of each face sample image in the face sample image pair correspond to N face key points in the predetermined face key point set;

[0149] The local feature of the corresponding face sample image is associated with n key point positions in the N key point positions of the corresponding face sample image;

[0150] The n key point positions of the corresponding face sample image correspond to n face key points in a predetermined subset of the predetermined face key point set; and

[0151] The n face key points are face key points in a local region of an unoccluded face.

[0152] 17. The apparatus of Appendix 14, wherein the weighted loss further comprises a pose similarity total loss for the face sample image set:

[0153] The pose similarity total loss for the face sample image set is a sum of pose feature similarity losses of pose feature pairs of face sample image pairs in the face sample image set;

[0154] The pose feature pair comprises a first pose feature for a corresponding unoccluded image and a second pose feature for a corresponding occluded image of the corresponding face sample image pair;

[0155] Wherein the first pose feature is a feature representing a head pose of the corresponding unoccluded image determined based on a global feature of the corresponding unoccluded image, and the second pose feature is a feature representing a head pose of the corresponding occluded image determined based on a global feature of the corresponding occluded image.

[0156] 18. The apparatus of Appendix 16, wherein the fused feature pair of the face sample image pair comprises a first fused feature and a second fused feature;

[0157] The first fused feature is determined by fusing a first global feature of an unoccluded image in the global feature pair of the face sample image pair and a first local feature of the unoccluded image;

[0158] The second fused feature is determined by fusing a second global feature of an occluded image in the global feature pair of the face sample image pair and a second local feature of the occluded image.

[0159] 19. The apparatus of Appendix 18, wherein a sum of a product of the first global feature and the first local feature and a fusion weight is taken as the first fused feature; and

[0160] A sum of a product of the second global feature and the second local feature and the fusion weight is taken as the second fused feature.

[0161] 20. A computer-readable storage medium having stored thereon a program, the program causing a computer to:

[0162] determine, for each face sample image pair in a face sample image set comprising unoccluded images and occluded images:

[0163] a global feature similarity loss for the face sample image pair based on a pair of global features of the face sample image pair;

[0164] a key point position deviation loss for the face sample image pair based on a pair of N key point positions of the face sample image pair; and

[0165] a classification loss for the face sample image pair based on a pair of fusion features of the face sample image pair;

[0166] determine a global feature similarity total loss, a key point position deviation total loss and a classification total loss for the face sample image set by summing;

[0167] determine a weighted loss by weighted summing the global feature similarity total loss, the key point position deviation total loss and the classification total loss;

[0168] determine whether the weighted loss converges or is less than a predetermined threshold; and

[0169] in a case where the determination result is "no", optimize the face recognition model by adjusting parameters of the face recognition model;

[0170] wherein the unoccluded image and the occluded image included in each face sample image pair are an unoccluded face sample image and an occluded face sample image of a same person;

[0171] each fusion feature in the pair of fusion features of the face sample image pair is a feature obtained by fusing a global feature and a local feature of a corresponding face sample image; and

[0172] the local feature of the corresponding face sample image is associated with an unoccluded face local region.

Claims

1. A method for training a face recognition model, the method comprising: Comprising: For each face sample image pair in the face sample image set, the face sample image pair comprises an unoccluded image and an occluded image: determining a global feature similarity loss of the face sample image pair based on a global feature pair of the face sample image pair; determining a key point position deviation loss of the face sample image pair based on an N key point position pair of the face sample image pair; and determining a classification loss of the face sample image pair based on a fusion feature pair of the face sample image pair; determining a global feature similarity total loss, a key point position deviation total loss and a classification total loss for the face sample image set by summation; determining a weighted loss by weighted summation of the global feature similarity total loss, the key point position deviation total loss and the classification total loss; determining whether the weighted loss converges or is less than a predetermined threshold; and optimizing the face recognition model by adjusting parameters of the face recognition model in the case of determining result being "no"; wherein the unoccluded image and the occluded image included in each face sample image pair are an unoccluded face sample image and an occluded face sample image of the same person; each fusion feature in the fusion feature pair of the face sample image pair is a feature obtained by fusing global features and local features of the corresponding face sample image; and the local features of the corresponding face sample image are associated with unoccluded face local regions.

2. The method of claim 1, wherein, N key points of each face sample image in the face sample image pair correspond to N face key points in a predetermined face key point set; the local features of the corresponding face sample image are associated with n key point positions in N key point positions of the corresponding face sample image; the n key point positions of the corresponding face sample image correspond to n face key points in a predetermined subset of the predetermined face key point set; and the n face key points are face key points within the unoccluded face local region.

3. The method of claim 1, wherein, the occluded face sample image is obtained by using an image processing module to add a cover covering part of the face to the face in the unoccluded face sample image.

4. The method of claim 1, wherein, the occluded image is a face image with a local region covered by a cover; and the cover is a mask, a scarf, a hat, an eye patch or sunglasses.

5. The method of claim 1, wherein, The weighted loss further comprises a pose similarity total loss for the face sample image set: the pose similarity total loss of the face sample image set is the sum of the pose feature similarity losses of the pose feature pairs of each face sample image pair in the face sample image set; the pose feature pair comprises a first pose feature of a corresponding unoccluded image in the corresponding face sample image pair and a second pose feature of a corresponding occluded image in the corresponding face sample image pair; the first pose feature is a feature representing a head pose of the corresponding unoccluded image determined based on global features of the corresponding unoccluded image; and the second pose feature is a feature representing a head pose of the corresponding occluded image determined based on global features of the corresponding occluded image.

6. The method of claim 2, wherein, the fusion feature pair of the face sample image pair comprises a first fusion feature and a second fusion feature; The first fusion feature is determined by fusing a first global feature of an unoccluded image and a first local feature of the unoccluded image in a pair of global features of the pair of face sample images; and The second fusion feature is determined by fusing a second global feature of an occluded image and a second local feature of the occluded image in the pair of global features of the pair of face sample images.

7. The method of claim 6, wherein, The first local feature of the unoccluded image is determined by: generating an intermediate feature by extracting a plurality of components from the first global feature of the unoccluded image using a window mask having n windows determined by n key point positions of the unoccluded image; determining a weight matrix by performing convolution processing on the intermediate feature; and determining a product of the first global feature and the weight matrix as the first local feature; wherein, in a case where the intermediate feature is aligned with the window mask, components of the intermediate feature located within each window of the window mask are equal to components at the same positions of the first global feature; and components of the intermediate feature located outside the n windows of the window mask are equal to zero or a predetermined small constant.

8. The method of claim 6, wherein, a sum of a product of the first global feature and the first local feature and a fusion weight is taken as the first fusion feature; and a sum of a product of the second global feature and the second local feature and the fusion weight is taken as the second fusion feature.

9. A method for recognizing a human face, characterized by, The method comprises: training a face recognition model using the method of claim 1; and performing face recognition on a face image to be recognized using the trained face recognition model.

10. A computer readable storage medium having stored thereon a program, characterized in that, The program causes a computer running the program to: for each pair of face sample images including an unoccluded image and an occluded image in a set of face sample images: determine a global feature similarity loss of the pair of face sample images based on a pair of global features of the pair of face sample images; determine a key point position deviation loss of the pair of face sample images based on a pair of N key point positions of the pair of face sample images; and determine a classification loss of the pair of face sample images based on a pair of fusion features of the pair of face sample images; determine a global feature similarity total loss, a key point position deviation total loss and a classification total loss for the set of face sample images by summation; determine a weighted loss by weighted summation of the global feature similarity total loss, the key point position deviation total loss and the classification total loss; determine whether the weighted loss converges or is less than a predetermined threshold; and in a case where the determination result is "no", optimize the face recognition model by adjusting parameters of the face recognition model; wherein the unoccluded image and the occluded image included in each pair of face sample images are an unoccluded face sample image and an occluded face sample image of the same person; each fusion feature in the pair of fusion features of the pair of face sample images is a feature obtained by fusing a global feature and a local feature of a corresponding face sample image; and the local feature of the corresponding face sample image is associated with an unoccluded face local region.

Images