Face recognition method and device

Abstract

A face recognition method and device, which can fuse facial pose features and facial features to fuse detailed information of the facial pose features and facial features, generate target facial features and facial pose information of a face image to be recognized based on the fused features, thereby improving the accuracy of the identified target facial features and facial pose information, and further improving the accuracy of the user recognition result identified based on the target facial features and facial pose information.

Description

[Technical field]

[0001] The present disclosure relates to the field of data processing technology, and in particular to a face recognition method and device. [Background technology]

[0002] Face recognition system is an emerging biometric technology, a high-precision technology currently being tackled in the international science and technology field, with broad development potential. Conventional face detection technologies perform well for faces in many ideal environments, but in some applications such as complex environments and cross-working scenarios, they have poor generalization ability and performance. In particular, in real application scenarios, especially when the face pose changes rapidly and dramatically, the face recognition accuracy of conventional methods is still far from the requirements of practical applications, and further research and improvement are needed. Summary of the Invention

[0003] In view of this, the embodiments of the present disclosure provide a face recognition method, an apparatus, a computer device and a computer-readable storage medium to solve the problem that face recognition results are inaccurate when face pose changes rapidly and abruptly in the prior art.

[0004] The first aspect of the disclosed embodiment is acquiring a face image to be recognized; Extracting facial pose features and facial features of a face image to be recognized; generating target facial feature and facial pose information of a face image to be recognized based on the facial pose feature and the facial feature; When the face pose information satisfies a predetermined condition, a user recognition result of the face image to be recognized is identified based on target facial features of the face image to be recognized.

[0005] A second aspect of the disclosed embodiment is an image acquisition unit for acquiring a face image to be recognized; a feature extraction unit for extracting facial pose features and facial features of a face image to be recognized; an information generating unit for generating target facial features, facial pose information, and corresponding score vectors of the facial pose information of a face image to be recognized according to the facial pose features and the facial features; A face recognition device is provided that includes a result identification unit for identifying a user recognition result for a face image to be recognized based on target facial features of the face image to be recognized when the face pose information and the corresponding score vector of the face pose information satisfy a predetermined condition.

[0006] In a third aspect of an embodiment of the present disclosure, there is provided a computing device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, the computing device implementing the steps of the method when the processor executes the computer program.

[0007] In a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the above method when executed by a processor.

[0008] Beneficial effects of the disclosed embodiment compared to the prior art: In the disclosed embodiment, the face image to be recognized can be obtained first, then the facial pose features and facial features of the face image to be recognized can be extracted, and then the target facial features and facial pose information of the face image to be recognized are generated based on the facial pose features and facial features, and when the facial pose information meets a predetermined condition, the user recognition result of the face image to be recognized is determined based on the target facial features of the face image to be recognized. In this embodiment, the facial pose features reflect the spatial information of the face and include facial structure features, edge features, and angle features, and the facial features reflect the features such as skin color and texture, so that when the facial pose features and facial features are fused, the detailed information of the facial pose features and facial features can be fused, and the target facial features and facial pose information of the face image to be recognized can be generated based on the fused features, so that the accuracy of the identified target facial features and facial pose information can be improved, and the accuracy of the user recognition result determined based on the target facial features and facial pose information can be improved. [Brief description of the drawings]

[0009] In order to more clearly describe the technical solutions in the embodiments of the present disclosure, the following briefly introduces drawings necessary for describing the embodiments or prior art. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can obtain other drawings based on these drawings without the need for creative work. [Figure 1] FIG. 2 is a scenario schematic diagram of an application scenario of an embodiment of the present disclosure. [Diagram 2] 1 is a flowchart of a face recognition method provided in an embodiment of the present disclosure. [Diagram 3] FIG. 2 is a block diagram of a face recognition device provided in an embodiment of the present disclosure. [Figure 4] FIG. 2 is a schematic diagram of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] In the following description, for the purposes of explanation, not limitation, specific details such as specific system structures, techniques, etc. are provided to thoroughly understand the embodiments of the present disclosure. However, those skilled in the art should understand that the present disclosure can be realized in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present disclosure with unnecessary details.

[0011] Hereinafter, a face recognition method and device according to an embodiment of the present disclosure will be described in detail with reference to the drawings.

[0012] In the prior art, the conventional face detection technology has good performance for faces in many ideal environments, but in some applications such as complex environments and cross-working scenarios, the generalization ability and performance to the environment are poor. In particular, in real application scenarios, especially when the face pose changes rapidly and dramatically, the face recognition accuracy of the conventional method is still far from the requirements of practical application, and further research and improvement are needed.

[0013] In order to solve the above problem, the present invention provides a face recognition method, in which the face image to be recognized can be first obtained, then the face pose features and face features of the face image to be recognized can be extracted, and then the target face features and face pose information of the face image to be recognized are generated based on the face pose features and face features, and when the face pose information meets a predetermined condition, the user recognition result of the face image to be recognized is determined based on the target face features of the face image to be recognized. In this embodiment, the face pose features reflect the spatial information of the face, and include face structure features, edge features, and angle features, and the face features reflect the features such as skin color and texture, so that the face pose features and face features are fused to generate the target face features and face pose information of the face image to be recognized, which can improve the accuracy of the determined target face features and face pose information, and can improve the accuracy of the user recognition result determined based on the target face features and face pose information.

[0014] By way of example, an embodiment of the invention can be applied to the application scenario depicted in Figure 1. This scenario may comprise a terminal device 1 and a server 2.

[0015] The terminal device 1 may be hardware or software. When the terminal device 1 is hardware, it may be various electronic devices having an image collection function and supporting communication with the server 2, including, but not limited to, a smartphone, a tablet, a laptop computer, and a desktop computer, and when the terminal device 1 is software, it may be installed in, for example, the electronic devices described above. The terminal device 1 may be realized as multiple software or software modules, or as a single software or software module, and the embodiment of the present disclosure is not limited thereto. The server 2 may be a server providing various services, such as a back-end server that receives a bill sent by a terminal device that establishes a communication connection with the server 1, and the back-end server can perform processing such as receiving and analyzing the bill sent by the terminal device to generate a processing result. The server 2 may be one server, a server cluster consisting of several servers, or a cloud computing service center, and the embodiment of the present disclosure is not limited thereto.

[0016] The server 2 may be hardware or software. If the server 2 is hardware, it may be various electronic devices that provide various services to the terminal device 1. If the server 2 is software, it may be multiple software programs or software modules that provide various services to the terminal device 1, or it may be a single software program or software module that provides various services to the terminal device 1, and the embodiment of the present disclosure is not limited to this.

[0017] The terminal device 1 and the server 2 may be communicatively connected via a network. The network may be a wired network connected using a coaxial cable, a twisted pair, or an optical fiber, or may be a wireless network capable of realizing interconnection of various communication devices without the need for wiring, such as Bluetooth, Near Field Communication (NFC), or Infrared, and the embodiments disclosed herein are not limited thereto.

[0018] Specifically, the user can input a face image to be recognized by the terminal device 1, and the terminal device 1 transmits the business data to be evaluated and the target evaluation method to the server 2. The server 2 first extracts the face pose features and face features of the face image to be recognized, and then generates target face features and face pose information of the face image to be recognized based on the face pose features and face features. When the face pose information satisfies a predetermined condition, the server 2 can specify the user recognition result of the face image to be recognized based on the target face features of the face image to be recognized, and returns the user recognition result of the face image to be recognized to the terminal device 1, so that the terminal device 1 displays the user recognition result of the face image to be recognized to the user. By fusing the face pose features and face features in this way, the detailed information of the face pose features and face features can be fusing, and the target face features and face pose information of the face image to be recognized can be generated based on the fused features, thereby improving the accuracy of the specified target face features and face pose information, and further improving the accuracy of the user recognition result specified based on the target face features and face pose information.

[0019] It should be noted that the specific types, numbers and combinations of the terminal device 1, the server 2 and the network can be adjusted according to the actual needs of the application scenario, and the embodiments disclosed herein are not limited thereto.

[0020] It should be noted that the above application scenarios are merely exemplified for easy understanding of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure may be used in any scenario of application.

[0021] Fig. 2 is a flowchart of a face recognition method provided in an embodiment of the present disclosure. The face recognition method in Fig. 2 may be implemented by the terminal device or the server in Fig. 1. As shown in Fig. 2, the face recognition method includes the following steps:

[0022] S201: A face image to be recognized is obtained.

[0023] In this embodiment, the facial image to be recognized can be understood as an image that needs facial recognition. For example, the facial image to be recognized can be collected by a surveillance camera installed at a fixed position, collected by a mobile terminal device, or read from a storage device in which the image is previously stored.

[0024] S202: Extract facial pose features and facial features of the face image to be recognized.

[0025] After obtaining the face image to be recognized, in order to accurately recognize the profile in the face image to be recognized, it is first necessary to extract the facial pose feature and facial feature of the face image to be recognized, where the facial pose feature can reflect the spatial information of the face, for example, including the facial structure feature, edge feature, and angle feature, and the facial feature can reflect the semantic information about the skin color, texture, age, lighting, race, etc. of the face.

[0026] S203: Based on the facial pose feature and facial feature, target facial feature and facial pose information of the face image to be recognized is generated.

[0027] In this embodiment, the facial pose feature and the facial feature are firstly fused, and the facial feature is supplemented with information by using the facial pose feature, so that the information of the facial feature can be enriched. Then, the target facial feature and the facial pose information of the face image to be recognized are generated based on the facial pose feature and the facial feature, so that the accuracy of the identified target facial feature and the facial pose information can be improved. Here, the target facial feature may be understood to include a feature vector of facial detail information (e.g., texture information, skin color information, etc.). The facial pose information may include a facial yaw angle, a pitch angle, and a roll angle, and the facial pose information may reflect a facial steering angle and a steering amplitude.

[0028] S204: If the face pose information satisfies a predetermined condition, a user recognition result for the face image to be recognized is specified based on the target face features of the face image to be recognized.

[0029] If the face is turned to the side or the head is lowered too much, the pose angle of the face image will be larger, and thus there will be less face information for recognition, making the face more vulnerable to attack (e.g., easier to decode using a photo of a user who is being impersonated). Therefore, in this embodiment, it is first necessary to determine whether the face pose information satisfies a predetermined condition, which in one implementation is that the yaw angle is less than or equal to a predetermined yaw angle threshold, the pitch angle is less than or equal to a predetermined pitch angle threshold, and the roll angle is less than or equal to a predetermined roll angle threshold.

[0030] When the face pose information meets a predetermined condition, the user recognition result of the face image to be recognized can be determined based on the target face features of the face image to be recognized.As an example, in this embodiment, some predetermined user information can be set in advance, and each predetermined user information has a corresponding predetermined user face feature, in this embodiment, the similarity between the target face features of the face image to be recognized and each of the predetermined user face features can be determined, for example, the vector distance can also be measured using methods such as Euclidean distance and cosine distance, and the similarity is determined based on the vector distance, and then the user information corresponding to the predetermined user face feature with the maximum similarity can be the user recognition result of the face image to be recognized.

[0031] Beneficial effects of the disclosed embodiment compared to the prior art: In the disclosed embodiment, the face image to be recognized can be obtained first, then the facial pose features and facial features of the face image to be recognized can be extracted, and then the target facial features and facial pose information of the face image to be recognized are generated based on the facial pose features and facial features, and when the facial pose information meets a predetermined condition, the user recognition result of the face image to be recognized is determined based on the target facial features of the face image to be recognized. In this embodiment, the facial pose features reflect the spatial information of the face and include facial structure features, edge features, and angle features, and the facial features reflect the features such as skin color and texture, so that when the facial pose features and facial features are fused, the detailed information of the facial pose features and facial features can be fused, and the target facial features and facial pose information of the face image to be recognized can be generated based on the fused features, so that the accuracy of the identified target facial features and facial pose information can be improved, and the accuracy of the user recognition result determined based on the target facial features and facial pose information can be improved.

[0032] Next, a method for implementing S202, that is, how to extract the facial pose features and facial features of the face image to be recognized, will be described. In this embodiment, the facial pose features can include N facial pose features, and the facial features can include N facial features, and S202 may include the following steps:

[0033] S202a: Regarding a first facial feature, a facial image to be recognized is input to a first convolutional layer to obtain the first facial feature.

[0034] S202b: For a first face pose feature, a face image to be recognized is input into a first residual block, and a first face pose feature is obtained.

[0035] S202c: For the pose feature of the ith face, input the pose feature of the (i-1)th face into the ith residual block to obtain the pose feature of the ith face.

[0036] S202d: For the i-th facial feature, input the i-1st face pose feature and the i-1st facial feature into the i-th convolutional layer to obtain the i-th facial feature, where i is greater than or equal to 2 and less than or equal to N, and N and i are positive integers.

[0037] In this embodiment, two models may be provided: a face feature extraction model and a face pose feature extraction model. Here, the face pose feature extraction model may include N residual blocks, which are at least a first residual block, a second residual block, ..., an Nth residual block, where these N residual blocks are cascaded; in one implementation, the face pose feature extraction model includes at least four residual blocks, each of which is composed of two, three, and two residual networks, respectively, and the network architecture of each residual network may include two convolution layers (i.e., conv), two batch normalization layers (i.e., bn), and two hyperbolic tangent activation functions (i.e., tanh), with a specific connection structure of conv+bn+tanh+conv+bn+tanh, and the channel numbers of the output feature maps of these three residual blocks are 64, 128, and 256, respectively. Note that the reason for using the hyperbolic tangent activation function as the activation function is that each feature calculation can take a value between (-1, 1) and contribute to the subsequent pose calculation. As can be seen, a face image to be recognized can be input into a first residual block to obtain the pose feature of the first face, the pose feature of the first face can be input into a second residual block to obtain the pose feature of the second face, ... for the pose feature of the i-th face, the pose feature of the (i-1)th face can be input into the i-th residual block to obtain the pose feature of the i-th face, where i is greater than or equal to 2 and less than or equal to N, and both N and i are positive integers.

[0038] Here, the facial feature extraction model may include N convolution layers, including at least a first convolution layer, a second convolution layer, ..., an Nth convolution layer, where these N convolution layers are cascaded. Here, the facial feature extraction model may be an IResNet-50 network, and each convolution layer may include one convolution operator with a convolution kernel size of 3x3 and a channel number of 192, and one convolution operator with a convolution kernel size of 1x1 and a channel number of 128. In this embodiment, for the first facial feature, a facial image to be recognized is input into a first convolutional layer to obtain the first facial feature, the pose feature of the first face (e.g., dimensions are (28, 28, 64)) and the first facial feature (e.g., dimensions are (28, 28, 128)) are input into a second convolutional layer (e.g., the first facial feature and the first facial feature are subjected to feature fusion, and then the fused feature is input into a second convolutional layer) to obtain the second facial feature, ..., the pose feature and the i-1th facial feature of the i-1th face are input into the ith convolutional layer to obtain the ith facial feature, where i is 2 or more and i is N or less, and both N and i are positive integers. It should be emphasized that the facial pose features are used as supplementary and input into the convolutional layer together with the facial features to calculate the next facial features. The reason is that the pose information takes into account more spatial information and can extract the facial structure features, edge features, and angle features relatively completely. On the other hand, the extraction of facial features needs to take into account various semantic information such as age, lighting, race, skin color, texture, etc., and there are certain deficiencies and semantic confusions in the spatial structure. Therefore, when the facial pose features extracted by the facial pose feature extraction model are added to the facial feature extraction model, it brings about an effective supplement to the information processing of facial features.

[0039] In addition, the number of residual blocks in the facial pose feature extraction model is smaller than the number of convolutional layers in the facial feature extraction model, and the number of channels is smaller. The reason for this is that the information processed by the facial pose feature extraction model is relatively single semantic information, and does not require a large amount of calculation.

[0040] Next, a description will be given of one implementation method of S203, that is, how to generate target facial features and facial pose information of a facial image to be recognized. In this embodiment, the step of "generating target facial features of a facial image to be recognized based on facial pose features and facial features" in S203 may include the following steps:

[0041] The pose feature and Nth facial feature of the Nth face are input to the N+1th convolutional layer to obtain the target facial features of the face image to be recognized.

[0042] In this embodiment, the facial feature extraction model further includes an N+1th convolutional layer, and the Nth face pose feature and the Nth facial feature can be input into the N+1th convolutional layer to obtain the target facial features of the facial image to be recognized.

[0043] In this embodiment, the step of "generating face pose information of the face image to be recognized based on the face pose features and the face features" in S203 may include the following steps.

[0044] Step a: Based on the facial pose features, generate a corresponding attention map of the facial pose features.

[0045] In this embodiment, downsampling process is performed on all face pose features, and the dimensions of all face pose features can be the same. Then, each face pose feature is input into the attention model to obtain the corresponding attention map of the face pose feature. Here, the attention model includes one convolution operator with a convolution kernel size of 1x1 and a channel number of 1, and a sigmoid function. For example, the dimension of one face pose feature is (28,28,64), and the face pose feature is first subjected to two convolution operations with a convolution kernel size of 3x3, a step size of 2, and a channel number of 64 to reduce the dimension to (7,7,64), and then the above 1x1 convolution operation and sigmoid operation are introduced to obtain the corresponding attention map as an attention map with a dimension of (7,7,1), and then the three dimensions are changed to one dimension to obtain the final attention map with a dimension of (7,7,1).

[0046] Step b: Based on the facial features, generate a corresponding attention map of the facial features.

[0047] In this embodiment, downsampling process is performed on all face features, and the dimensions of all face features can be the same. Then, each face feature is input into the attention model to obtain the corresponding attention map of the face feature. Here, the attention model includes one convolution operator with convolution kernel size 1x1 and channel number 1, and a sigmoid function. For example, the dimension of one face feature is (28,28,64), and the convolution operation with convolution kernel size 3x3, step size 2, and channel number 64 is first performed twice on the face feature to reduce the dimension to (7,7,64), and then the above 1x1 convolution operation and sigmoid operation are introduced to obtain the corresponding attention map as an attention map with dimension (7,7,1), and then the three dimensions are changed to one dimension to obtain the final attention map dimension as (7,7,1).

[0048] Step c: Obtain a full-space attention map based on the corresponding attention map of the facial pose features, the corresponding attention map of the facial features and the N facial pose features.

[0049] In this embodiment, first, a second-stream attention map can be generated based on the corresponding attention map of the face pose feature and the corresponding attention map of the face feature. For example, the i-th second-stream attention map can be generated based on the corresponding attention map of the i-th face pose feature and the corresponding attention map of the i-th face feature. Specifically, the i-th second-stream attention map D i can be calculated, and D i =reshape([A,B] T W1), where A is the corresponding attention map of facial features, B is the corresponding attention map of facial pose features, W1 is a learning parameter matrix with matrix dimensions (7x7x2,qx7x7), each row of W1 represents the association value between each point and other points in space, but such associations can take various forms, so we introduce q types of association learning, and reshape() is a function that converts a matrix into a matrix of a specific dimension.

[0050] Then, a dark and shallow attention map can be generated based on the corresponding attention maps of the facial features. A dark and shallow attention map can be generated based on the corresponding attention maps of all the facial features. Specifically, the dark and shallow attention map E can be calculated by the following formula: E=reshape([A1,A2,...,A N ] T W2), reshape() is a function that transforms a matrix into a matrix of a certain dimension, A is the corresponding attention map of a facial feature, N is the number of facial features, W2 is the learning parameter matrix, and W 2の The matrix dimensions are (7*7*2,q'*q), and the parameter q' may be set to 4, where q' also acts as a partition, and the dimensions of E are (q',q).

[0051] Then, a full-space attention map is obtained based on the two-stream attention map, the dark and shallow attention map, and the N facial pose features. For example, the full-space attention map R can be calculated by the following formula: R=(E T D1+E T D2+…+E T D N ) T P N where E is the dark and shallow attention map and D i is the i-th stream attention map, 1≦i≦N, and P N is the Nth face pose feature.

[0052] Step d: Based on the full-space attention map, generate face pose information of the face image to be recognized.

[0053] In this embodiment, the facial pose information of the facial image to be recognized and the corresponding score vector of the facial pose information can be generated based on the full space attention map. The corresponding score vector of the facial pose information is for evaluating the richness of the overall facial information (discriminative information contained in the facial photo), and as can be seen, the higher the score vector, the richer the overall facial information, and conversely, the lower the score vector, the less rich the overall facial information. Specifically, the facial pose information O of the facial image to be recognized can be calculated by the following formula: O=sigmoid((relu(R T W3) T W4), where R is the full-space attention map, W3 and W4 are parameter matrices, and the dimensions of W3 and W4 are (256, 128), (128, 1), respectively.

[0054] As can be seen, various forms of full-space attention information are fused into the full-space attention map, and deep pose and shallow pose (i.e., dark and shallow attention map) and overall face feature information (i.e., Nth face pose feature) are also fused, so that the facial pose information of the face image to be recognized generated based on the full-space attention map and the corresponding score vector of the facial pose information are more accurate.

[0055] Correspondingly, when the face pose information satisfies a predetermined condition, the step of determining a user recognition result of the face image to be recognized based on the target face features of the face image to be recognized may include:

[0056] When the face pose information and the score vector corresponding to the face pose information satisfy a predetermined condition, a user recognition result of the face image to be recognized is specified based on the target face features of the face image to be recognized.

[0057] Here, the predetermined condition may be that the yaw angle is equal to or less than a predetermined yaw angle threshold, the pitch angle is equal to or less than a predetermined pitch angle threshold, the roll angle is equal to or less than a predetermined roll angle threshold, and the score vector corresponding to the face pose information is equal to or greater than a predetermined score. Note that if the score vector corresponding to the face pose information is smaller than a predetermined score, the face becomes vulnerable to attack (e.g., it is easy to decrypt using a photo of a user who is being impersonated), so the score vector corresponding to the face pose information needs to be equal to or greater than a predetermined score.

[0058] It should be noted that the above embodiment can be applied to a face recognition model, where the training process of the face recognition model is described as follows.

[0059] In this embodiment, N class centers (one positive class center and N-1 negative class centers), i.e., N vectors, may be introduced. These vectors and the face vector T (after normalization) are point-wise multiplied to obtain N numbers, which represent the similarity x between the current face and the N class centers. A common training method is to perform a softmax operation on these N x's, and then calculate the cross-entropy. However, such a decision boundary is not accurate, and the training method is also inefficient. Therefore, in this embodiment, the similarity between T and the positive class center is x + and x + needs to be subtracted from the decision value related to the pose,evaluation value. Subtracting a larger decision value means that the decision boundary of the facial feature is smaller.,Now, if we set the yaw angle as y, the pitch angle as p, the roll angle as r, the evaluation value as s, and the decision value as m, the general formula is, x + =x + -m, JPEG2025502745000002.jpg49170As above, f is a function, and m0 and m1 are hyper-parameters, for example m0=0.2, m1=0.2.

[0060] The above formula also has one value i that determines which face is currently given a larger decision value and a smaller decision boundary. In general, the larger m is, the tighter the decision boundary is, the closer the facial features are to the class center, and the larger the gradient generated. When i is set to 0, the above formula gives a larger decision value, a smaller decision boundary, and a larger gradient to faces with very small yaw, pitch, and roll angles and very high evaluation values, and as i gradually increases, the above formula gives a larger decision value, a smaller decision boundary, and a larger gradient to faces with relatively large yaw, pitch, and roll angles and relatively low evaluation values. That is, when i takes the value 0, the network mainly trains forward-looking faces and gives forward-looking face gradients, and as i gradually increases, the network mainly trains profile faces and gives profile face gradients.

[0061] Technical solution for deciding on training of face recognition model: When the model starts training, set i to 0. After that, the model loss will gradually decrease. If the accuracy of the model gradually improves on the validation set, i can be gradually increased by 0.1 each time, such as from 0 to 0.1, from 0.1 to 0.2, etc. Continue to observe the accuracy on the validation set. If it is found that the accuracy has decreased, decrease i by 0.1, and try to return i to the original value after a period of training. Repeat increasing to the original value three times, but if the accuracy cannot continue to improve, decrease it by 0.1 and end training after fitting. At this time, the spatial distribution of the obtained face shows a suitable distribution in which the angle is small near the center of the class and the angle is large at the edge of the class.

[0062] Due to the reasonable spatial distribution, the inference process can obtain better feature representation of the face in large poses, and can also directly obtain the three pose angles and evaluation values, which makes the face comparison more flexible, and can stop the comparison when the pose angles of the two photos in the face comparison are too large.

[0063] All the above-mentioned optional technical solutions may be combined in any combination to form optional embodiments of the present disclosure, and will not be described one by one here.

[0064] Below are apparatus embodiments of the present disclosure for carrying out the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0065] FIG. 3 is a schematic diagram of a face recognition device provided in an embodiment of the present disclosure. As shown in FIG. 3, the face recognition device includes: an image acquisition unit 301 for acquiring a face image to be recognized; a feature extraction unit 302 for extracting facial pose features and facial features of a face image to be recognized; an information generating unit 303 for generating target facial features, facial pose information, and corresponding score vectors of the facial pose information of a face image to be recognized according to the facial pose features and the facial features; and a result determination unit 304 for determining a user recognition result for the face image to be recognized based on target facial features of the face image to be recognized when the face pose information and the corresponding score vector of the face pose information satisfy a predetermined condition.

[0066] Preferably, the facial pose features comprise N facial pose features, the facial features comprise N facial features, and the feature extraction unit 302 comprises: Regarding the first facial feature, a facial image to be recognized is input to a first convolutional layer to obtain the first facial feature; For a first face pose feature, input a face image to be recognized into a first residual block to obtain a first face pose feature; for a pose feature of the i-th face, inputting the pose feature of the (i-1)-th face into the i-th residual block to obtain the pose feature of the i-th face; For the i-th facial feature, the i-1st face pose feature and the i-1st facial feature are input into the i-th convolutional layer to obtain the i-th facial feature, where i is greater than or equal to 2 and less than or equal to N, and N and i are positive integers.

[0067] Preferably, the information generating unit 303 comprises: The pose feature and Nth facial feature of the Nth face are input to the N+1th convolutional layer to obtain the target facial features of the face image to be recognized.

[0068] Preferably, the information generating unit 303 comprises: generating a corresponding attention map of the facial pose features based on the facial pose features; generating, based on the facial features, a corresponding attention map of the facial features; and obtaining a full space attention map based on the corresponding attention map of the facial pose features, the corresponding attention map of the facial features, and the N facial pose features; This is to generate face pose information of a face image to be recognized based on a full-space attention map.

[0069] Preferably, the information generating unit 303 specifically comprises: generating a two-stream attention map based on the corresponding attention map of facial pose features and the corresponding attention map of facial features; for generating dark and shallow attention maps based on corresponding attention maps of facial features; Based on the two-stream attention map, the dark and shallow attention map and the N facial pose features, a full-space attention map is obtained.

[0070] Preferably, the information generating unit 303 specifically comprises: generating face pose information of a face image to be recognized based on the full-space attention map, and a corresponding score vector of the face pose information; Correspondingly, the result determination unit 304: When the facial pose information and the score vector corresponding to the facial pose information satisfy a predetermined condition, a user recognition result of the facial image to be recognized is specified based on the target facial features of the facial image to be recognized.

[0071] Preferably, the facial pose information includes a yaw angle, a pitch angle, and a roll angle, and the predetermined conditions are that the yaw angle is less than or equal to a predetermined yaw angle threshold, the pitch angle is less than or equal to a predetermined pitch angle threshold, the roll angle is less than or equal to a predetermined roll angle threshold, and the corresponding score vector of the facial pose information is greater than or equal to a predetermined score.

[0072] Preferably, the result determination unit 304 comprises: A method for identifying a similarity between a target facial feature of a facial image to be recognized and each of the predetermined user facial features; This is to determine the user information corresponding to the predetermined user facial feature with the greatest similarity as the facial image-based user recognition result to be recognized.

[0073] The technical solution provided in the disclosed embodiment is a face recognition device including an image acquisition unit for acquiring a face image to be recognized, a feature extraction unit for extracting facial pose features and facial features of the face image to be recognized, an information generation unit for generating target face features, facial pose information, and a corresponding score vector of the facial pose information of the face image to be recognized based on the facial pose features and facial features, and a result identification unit for identifying a user recognition result of the face image to be recognized based on the target face features of the face image to be recognized when the facial pose information and the corresponding score vector of the facial pose information satisfy a predetermined condition. In this embodiment, the facial pose features reflect the spatial information of the face, and include facial structure features, edge features, and angle features, and the facial features reflect features such as facial skin color and texture. Therefore, by fusing the facial pose features and facial features, detailed information of the facial pose features and facial features can be fused, and target facial features and facial pose information of the face image to be recognized can be generated based on the fused features, thereby improving the accuracy of the identified target facial features and facial pose information, and improving the accuracy of the user recognition results identified based on the target facial features and facial pose information.

[0074] It should be understood that the size of the numbers of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and does not arbitrarily limit the implementation process of the embodiments disclosed herein.

[0075] 4 is a schematic diagram of a computer device 4 provided in an embodiment of the present disclosure. As shown in FIG. 4, the computer device 4 of the embodiment includes a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable by the processor 401. When the processor 401 executes the computer program 403, it realizes the steps in each of the above method embodiments. Or, when the processor 401 executes the computer program 403, it realizes the functions of each module / unit in each of the above device embodiments.

[0076] For illustration, the computer program 403 may be divided into one or more modules / units, and one or more modules / units may be stored in the memory 402 and executed by the processor 401 to accomplish the present disclosure. One or more modules / units may be a series of computer program command sections capable of performing a specific function, and the command sections are for explaining the process of the computer program 403 being executed in the computer device 4.

[0077] The computing device 4 may be a computing device such as a desktop computer, a laptop, a palmtop computer, a cloud server, etc. The computing device 4 may include, but is not limited to, a processor 401 and a memory 402. As will be appreciated by those skilled in the art, FIG. 4 is merely an example of the computing device 4, and is not intended to limit the computing device 4, and may include more or fewer components than shown, or may combine certain components or different components, for example, the computing device may include input / output devices, network access devices, buses, etc.

[0078] The processor 401 may be a Central Processing Unit (CPU) or other general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The general purpose processor may be a microprocessor, any common processor, etc.

[0079] The memory 402 may be an internal storage unit of the computer equipment 4, such as a hard disk or memory of the computer equipment 4. The memory 402 may be an external storage device of the computer equipment 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc., that is provided in the computer equipment 4. Moreover, the memory 402 may comprise an internal storage unit of the computer equipment 4 or an external storage device. The memory 402 is for storing computer programs and other programs and data required by the computer equipment. The memory 402 may also be used to temporarily store data that has been output or is to be output.

[0080] For convenience and simplicity of explanation, those skilled in the art can clearly understand that, although only the division of each of the above functional units and modules is given as an example, in actual application, the above functions can be assigned to different functional units and modules as necessary, that is, all or part of the above-described functions can be completed by dividing the internal structure of the device into different functional units or modules. Each functional unit and module in the embodiment may be integrated into one processing unit, each unit may exist physically alone, or two or more units may be integrated into one unit, and the integrated unit may be realized in the form of hardware or software functional unit. Note that the specific names of each functional unit and module are provided for easy distinction from each other, and do not limit the scope of protection of the present disclosure. For the specific operation processes of the units and modules in the above system, reference may be made to the corresponding processes in the embodiments of the above method, and the description will be omitted here.

[0081] In the above embodiments, the description of each embodiment has its own emphasis, and for the details or parts not described in an embodiment, reference may be made to the relevant descriptions of other embodiments.

[0082] Those skilled in the art can be aware that each example unit and algorithm step described in the embodiments disclosed herein can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented by hardware or software is determined by the specific application and design constraints of the technical solution. Those skilled in the art may implement the described functions using different methods for each specific application, but such implementation should not be considered to go beyond the scope of this disclosure.

[0083] In the embodiments provided in this disclosure, it should be understood that the disclosed device / computer device and method can be realized in other ways. For example, the above-described device / computer device embodiments are merely illustrative, and the division into modules or units is merely a logical functional division, and other division methods may be applied in actual implementation, for example, multiple units or components may be combined or integrated into other systems, or some features may be ignored or not implemented. In addition, the couplings or direct couplings or communication connections between the displayed or discussed interfaces may be indirect couplings or communication connections through some interfaces, devices or units, and may be electrical, mechanical, or other forms.

[0084] Units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, i.e., located in one place or distributed among multiple network units, some or all of which may be selected according to actual needs to achieve the objectives of the solutions of the present embodiment.

[0085] In addition, each functional unit in this disclosure may be integrated into one processing unit, each unit may exist physically alone, or two or more units may be integrated into one unit. The integrated unit may be realized in the form of hardware or a software functional unit.

[0086] The integrated module / unit may be realized in the form of a software functional unit and stored in a computer-readable storage medium when sold or used as an independent product. With this understanding, the present disclosure can realize all or part of the processes in the method of the above embodiments by commanding relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of each of the above method embodiments can be realized. The computer program may include a computer program code, and the computer program code may be in source code form, object code form, executable file, or some intermediate form, etc. The computer-readable storage medium may include any entity or device capable of carrying a computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a Read-Only Memory (ROM), a Random Access Memory (RAM), an electric carrier wave signal, an electric communication signal, and a software distribution medium, etc. Furthermore, the content contained on a computer-readable storage medium may be increased or decreased as required by the legislation and patent practice of a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, a computer-readable storage medium does not include electrical carrier signals and telecommunications signals.

[0087] The above-mentioned embodiments are only for illustrating the technical solutions of the present disclosure, but are not intended to limit the same. Although the present disclosure has been described in detail with reference to the above embodiments, those skilled in the art may still amend the technical solutions described in the above embodiments, or equivalently replace some of the technical features therein, but it should be understood that such amendments or replacements do not deviate from the substance of the corresponding technical solutions from the idea and scope of the technical solutions of the embodiments of the present disclosure, and all of them should be included in the protection scope of the present disclosure.

Claims

1. 1. A face recognition method, comprising: acquiring a face image to be recognized; extracting facial pose features and facial features of the face image to be recognized; generating target facial feature and facial pose information of the face image to be recognized based on the facial pose feature and the facial feature; specifying a user recognition result of the face image to be recognized based on target facial features of the face image to be recognized when the face pose information satisfies a predetermined condition; A face recognition method comprising:

2. The facial pose features include N facial pose features, and the facial features include N facial features, and the step of extracting the facial pose features and facial features of the facial image to be recognized includes: For a first facial feature, inputting the facial image to be recognized into a first convolutional layer to acquire the first facial feature; For a first face pose feature, input the face image to be recognized into a first residual block to obtain a first face pose feature; For the pose feature of the i-th face, input the pose feature of the (i-1)-th face into the i-th residual block to obtain the pose feature of the i-th face; For the i-th facial feature, input the i-1-th face pose feature and the i-1-th facial feature into the i-th convolutional layer to obtain the i-th facial feature, where i is greater than or equal to 2 and less than or equal to N, and N and i are positive integers.

2. The method of claim 1 .

3. The step of generating target facial feature and facial pose information of the face image to be recognized based on the facial pose feature and the facial feature includes: inputting the Nth face pose feature and the Nth face feature into an N+1th convolutional layer to obtain target face features of the face image to be recognized; 3. The method of claim 2 .

4. The step of generating face pose information of the face image to be recognized based on the face pose features and the face features includes: generating a corresponding attention map of the facial pose features based on the facial pose features; generating a corresponding attention map for the facial features based on the facial features; obtaining a full space attention map based on the corresponding attention map of the facial pose feature, the corresponding attention map of the facial feature, and the N facial pose feature; generating face pose information of the face image to be recognized based on the full space attention map.

3. The method of claim 2 .

5. The step of obtaining a full space attention map based on the corresponding attention map of the facial pose feature, the corresponding attention map of the facial feature, and the N facial pose feature includes: generating a two-stream attention map based on the corresponding attention map of the facial pose features and the corresponding attention map of the facial features; generating a dark and shallow attention map based on the corresponding attention map of the facial features; and obtaining a full-space attention map based on the two-stream attention map, the dark and shallow attention map, and the N facial pose features.

5. The method of claim 4.

6. The step of generating face pose information of the face image to be recognized based on the full space attention map includes: generating face pose information of the face image to be recognized based on the full space attention map, and a score vector corresponding to the face pose information; Correspondingly, when the face pose information satisfies a predetermined condition, the step of identifying a user recognition result of the face image to be recognized based on target face features of the face image to be recognized includes: and when the face pose information and the score vector corresponding to the face pose information satisfy a predetermined condition, specifying a user recognition result of the face image to be recognized based on target face features of the face image to be recognized.

5. The method of claim 4.

7. The face pose information includes a yaw angle, a pitch angle, and a roll angle, and the predetermined condition is that the yaw angle is equal to or less than a predetermined yaw angle threshold, the pitch angle is equal to or less than a predetermined pitch angle threshold, the roll angle is equal to or less than a predetermined roll angle threshold, and a score vector corresponding to the face pose information is equal to or greater than a predetermined score.

7. The method of claim 6.

8. The step of identifying a user recognition result of the facial image to be recognized based on target facial features of the facial image to be recognized includes: Identifying a similarity between a target facial feature of the face image to be recognized and each of the predetermined user facial features; and determining the user information corresponding to the predetermined user facial feature having the greatest similarity as a facial image-based user recognition result to be recognized.

2. The method of claim 1 .

9. A face recognition device, comprising: The apparatus includes an image acquisition unit for acquiring a face image to be recognized, and a feature extraction unit for extracting face pose features and face features of the face image to be recognized; an information generating unit for generating target facial features, facial pose information, and a corresponding score vector of the facial pose information of the face image to be recognized according to the facial pose features and the facial features; and a result determination unit for determining a user recognition result of the face image to be recognized based on target face features of the face image to be recognized when the face pose information and the score vector corresponding to the face pose information satisfy a predetermined condition. A face recognition device comprising:

10. A computing device comprising a memory, a processor, and a computer program stored in said memory and executable by said processor, said computing device performing the steps of the method of claim 1 when said processor executes said computer program.

1. A computer device comprising:

11. A computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the method according to claim 1 when executed by a processor. A computer-readable storage medium comprising:

Images