Face recognition methods, devices, equipment and computer-readable storage media

Abstract

This application provides a face recognition method, apparatus, device, and computer-readable storage medium. The method includes acquiring an image of the upper half of a user's face to be recognized; inputting the upper half of the face image into a face recognition model to obtain image features; wherein, the face recognition model is obtained by training a first convolutional neural network model based on a set of upper half-face images; during model training, not only the loss function of the first convolutional neural network model itself is considered, but also the loss function formed between the first convolutional neural network model and a second convolutional neural network model, thereby updating the network parameters of the first convolutional neural network model to obtain the face recognition model; comparing the image features with face features in a face database, and determining whether the face recognition of the user to be recognized passes based on the comparison result. This method can improve the accuracy of face recognition and can identify users wearing masks or not wearing masks, making it very convenient to use.

Description

Technical Field

[0001] This application relates to the field of facial recognition technology, specifically to a facial recognition method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Facial recognition is a biometric technology that identifies individuals based on their facial features. It is widely used in many fields such as attendance, access control, and security checks. Facial recognition typically uses machine learning methods to train a facial recognition model, and then uses this model to identify faces in images.

[0003] Current facial recognition typically extracts the entire facial features and then associates these features with the user for identity verification and authentication. However, during certain periods, users wear masks, which obscure the lower half of their face (such as the mouth and nose). In such cases, using current training models for facial recognition often results in poor recognition performance, and may even lead to failure to recognize the face altogether. Summary of the Invention

[0004] In view of this, this application provides a face recognition method, apparatus, device, and computer-readable storage medium.

[0005] In a first aspect, embodiments of this application provide a face recognition method, the method comprising:

[0006] Obtain an image of the upper half of the user's face to be identified;

[0007] The upper half of the face image is input into a face recognition model to obtain image features;

[0008] The face recognition model is obtained by training the first convolutional neural network model based on a set of upper half face images.

[0009] In the training of the first convolutional neural network model, the network parameters of the first convolutional neural network model are updated using the first loss function and the second loss function as the total loss function until the model training is completed, thus obtaining the face recognition model.

[0010] The first loss function is the cross-entropy loss function formed by the features of the upper half face image samples extracted by the first convolutional neural network model and the sample labels of the upper half face image samples; wherein, the performance of the second convolutional neural network model is better than that of the first convolutional neural network model.

[0011] The second loss function is a mean squared loss function formed by extracting different features of the upper half face image samples using the first convolutional neural network model and the second convolutional neural network model, respectively.

[0012] The image features are compared with facial features in the face database, and the facial recognition of the user to be identified is determined based on the comparison results.

[0013] Secondly, embodiments of this application provide a face recognition device, which includes...

[0014] The upper half face image acquisition module is used to acquire the upper half face image of the user to be identified;

[0015] The image feature acquisition module is used to input the upper half face image into the face recognition model to obtain image features;

[0016] The face recognition model is obtained by training the first convolutional neural network model based on a set of upper half face images.

[0017] In the training of the first convolutional neural network model, the network parameters of the first convolutional neural network model are updated using the first loss function and the second loss function as the total loss function until the model training ends, thus obtaining the face recognition model; wherein, the performance of the second convolutional neural network model is better than that of the first convolutional neural network model.

[0018] The first loss function is the cross-entropy loss function formed by the features of the upper half face image samples extracted by the first convolutional neural network model and the sample labels of the upper half face image samples;

[0019] The second loss function is a mean squared loss function formed by extracting different features of the upper half face image samples using the first convolutional neural network model and the second convolutional neural network model respectively; the comparison module is used to compare the image features with the face features in the face database, and determine whether the face recognition of the user to be identified passes based on the comparison results.

[0020] Thirdly, embodiments of this application provide a terminal device, including: a memory; one or more processors coupled to the memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to perform the face recognition method provided in the first aspect above.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, which can be called by a processor to execute the face recognition method provided in the first aspect.

[0022] The face recognition method, apparatus, device, and computer-readable storage medium provided in this application first acquire an image of the upper half of the user's face to be recognized; the upper half of the face image is input into a face recognition model to obtain image features; wherein, the face recognition model is obtained by training a first convolutional neural network model based on an upper half of the face image set; during the training of the first convolutional neural network model, the network parameters of the first convolutional neural network model are updated using a first loss function and a second loss function as the total loss function until the model training ends, thus obtaining the face recognition model; the first loss function is a cross-entropy loss function formed by the features of the upper half of the face image samples extracted by the first convolutional neural network model and the sample labels of the upper half of the face image samples; the second loss function is a mean square loss function formed by the different features of the upper half of the face image samples extracted by the first convolutional neural network model and the second convolutional neural network model respectively; the image features are compared with the face features in the face database, and the face recognition of the user to be recognized is determined based on the comparison result.

[0023] This face recognition method uses a face recognition model to extract image features from the upper half of a user's face. The face recognition model is trained on a first convolutional neural network (CNN) model using a set of upper half-face images. During model training, not only the loss function of the first CNN model itself is considered, but also the loss function formed between the first and second CNN models. This is used to update the network parameters of the first CNN model. In other words, the characteristics of both the first and second CNN models are used to train the first CNN, resulting in a face recognition model that possesses characteristics of both models. Furthermore, the second CNN model outperforms the first CNN model, thus improving the accuracy of face recognition. This method can identify users wearing masks as well as those not wearing masks, making it very convenient to use. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0025] Figure 1 This is a schematic diagram illustrating an application scenario of the face recognition method provided in the embodiments of this application;

[0026] Figure 2A flowchart illustrating a face recognition method provided in one embodiment of this application;

[0027] Figure 3 A schematic diagram illustrating the training of a face recognition model with features provided in one embodiment of this application;

[0028] Figure 4 A structural diagram of an improved MobileFaceNet model provided in one embodiment of this application;

[0029] Figure 5 This is a schematic diagram of the structure of MS_CAM provided in one embodiment of this application;

[0030] Figure 6 A schematic diagram of the structure of an improved ResNet model provided in one embodiment of this application;

[0031] Figure 7 This is a schematic diagram of the structure of an AFF provided in one embodiment of this application;

[0032] Figure 8 This is a structural diagram of a face recognition device provided in one embodiment of this application;

[0033] Figure 9 This is a schematic diagram of the structure of a terminal device provided in one embodiment of this application;

[0034] Figure 10 This is a schematic diagram of the structure of a computer-readable storage medium provided in one embodiment of this application. Detailed Implementation

[0035] The technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0036] To provide a more detailed description of this application, the following description, in conjunction with the accompanying drawings, details a face recognition method, apparatus, terminal device, and computer storage medium provided in this application.

[0037] Please refer to Figure 1 , Figure 1This diagram illustrates an application scenario of the face recognition method provided in this application embodiment. The application scenario includes the terminal device 100 provided in this application embodiment. The terminal device 100 can be various electronic devices with a display screen (such as structural diagrams 102, 104, 106, and 108), including but not limited to smartphones, computer devices, time clocks, and attendance machines. The computer device can be at least one of desktop computers, portable computers, laptop computers, and tablet computers. When a user's face image is captured, the terminal device 100 executes the face recognition method of this application. For details, please refer to the face recognition method embodiment. Specifically, the acquisition component of the terminal device 100 can be used to capture the user's face image with the user's authorization. For example, in a judicial setting, the terminal device 100 can be used to capture the face image of an authorized user, and then the face recognition method of this application can be used for user registration. Another example is that, with user authorization, an attendance machine can capture the face image of an authorized user to identify the authorized user and register attendance.

[0038] The terminal equipment (e.g., time and attendance machine) provided in this application can be applied to the field of public safety maintenance. For example, it can be applied to public safety places such as enterprises and residential communities to provide a certain level of security for enterprise personnel and community residents. When using this application, indicator signs can be set up in specific application scenarios to remind users that their facial images will be collected when using the time and attendance machine. With user authorization, the facial images of users can be legally collected to achieve the facial recognition functionality described in this application.

[0039] Secondly, terminal device 100 can refer to one of multiple terminal devices; this embodiment only uses terminal device 100 as an example. Those skilled in the art will understand that the number of terminal devices can be more or less. For example, there may be only a few terminal devices, or dozens or hundreds, or even more. This application embodiment does not limit the number or type of terminal devices. Terminal device 100 can be used to execute a face recognition method provided in this application embodiment.

[0040] In one optional implementation, the application scenario may include not only the terminal device 100 provided in this embodiment, but also a server, wherein a network is established between the server and the terminal device. The network serves as a medium for providing a communication link between the terminal device and the server. The network may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0041] It should be understood that the number of terminal devices, networks, and servers is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be used. For example, the server can be a server cluster composed of multiple servers. The terminal devices interact with the server through the network to receive or send messages, etc. The server can be a server providing various services. The server can be used to execute the steps of the face recognition method provided in this application embodiment. Furthermore, when executing the face recognition method provided in this application embodiment, some steps can be executed on the terminal device and some steps can be executed on the server; this is not limited here.

[0042] Based on this, this application provides a face recognition method in its embodiments. Please refer to... Figure 2 , Figure 2 This illustration shows a flowchart of a face recognition method provided in an embodiment of this application, demonstrating its application in... Figure 1 Taking a terminal device as an example, the explanation includes the following steps:

[0043] Step S110: Obtain the upper half face image of the user to be identified.

[0044] The upper half of the user's face image refers to the upper half of the face image of any user who needs to undergo face recognition. This upper half face image can be extracted from the user's entire face image. The user's entire face image can be captured using the image acquisition module built into the terminal device or using a separate image acquisition device.

[0045] Optionally, the image of the user's entire face can be a masked full-face image or an unmasked full-face image, such as a face image with or without a mask.

[0046] An upper face image typically refers to an image that includes information such as the eyes, forehead, eyebrows, and hair.

[0047] The specific process of extracting the upper half of the face image from the user's entire face image is as follows: Align the user's entire face image to a preset size A (e.g., 112*112), then divide the face equally along the vertical line on the height side of the image, take the upper half of the face, and then scale it to the preset size A.

[0048] Step S120: Input the upper half of the face image into the face recognition model to obtain image features.

[0049] The face recognition model is obtained by training a first convolutional neural network model based on a set of upper face images. During the training of the first convolutional neural network model, the network parameters are updated using a first loss function and a second loss function as the total loss function until training is complete, resulting in the face recognition model. The performance of the second convolutional neural network model is superior to that of the first model. The first loss function is the cross-entropy loss function formed by combining the features extracted from the upper face image samples in the set by the first convolutional neural network model with the sample labels of the upper face image samples. The second loss function is the mean square loss function formed by extracting different features from the upper face image samples using the first and second convolutional neural network models, respectively.

[0050] In one embodiment, the first convolutional neural network model and the second convolutional neural network model include a ResNet network model, a VGG network model, a DenseNet network model, a SqueezeNet network model, or a MobileFaceNet network model.

[0051] Among them, ResNet, VGG, DenseNet, SqueezeNet, and MobileFaceNet are all very useful convolutional neural network models that can be used for feature extraction in face recognition.

[0052] It should be noted that the first convolutional neural network model and the second convolutional neural network model are usually different models, and the performance of the second convolutional neural network model is better than that of the first convolutional neural network model.

[0053] In addition, model training involves giving an input vector and a target output value, then inputting the input vector into one or more network structures or functions to obtain the actual output value. Based on the target output value and the actual output value, biases are calculated, and it is determined whether the biases are within acceptable limits. If they are within acceptable limits, training ends and the relevant parameters are fixed. If they are not within acceptable limits, some parameters in the network structure or function are continuously adjusted until the biases are within acceptable limits or a certain termination condition is met. At this point, training ends and the relevant parameters are fixed. Finally, the trained model is obtained based on the fixed parameters.

[0054] In this embodiment, the training of the face recognition model is actually as follows: image samples from the upper half of the face image set are used to train the first convolutional neural network model and the second convolutional neural network model. During model training, the network parameters of the first convolutional neural network model are updated, but the network parameters of the second convolutional neural network model are not updated. That is, in the training of the first convolutional neural network model, the first convolutional neural network model is used to process the upper half of the face image samples, and the cross-entropy loss function formed by the extracted features and the sample labels of the upper half of the face image samples, as well as the mean square loss function formed by the different features extracted from the first and second convolutional neural network models, are used as the total loss function to update the network parameters of the first convolutional neural network model. When the network parameters are optimal, the face recognition model is obtained.

[0055] Furthermore, a specific implementation method for training a face recognition model is given and described as follows:

[0056] In one embodiment, the face recognition model is obtained through the following method:

[0057] S1: Obtain the upper half face image set, which includes upper half face image samples labeled with sample tags.

[0058] The upper half face image collection stores a series of upper half face image samples from users, which are usually labeled with sample tags.

[0059] Next, an implementation method for generating an upper half face image set is given and described as follows:

[0060] In one embodiment, obtaining an upper half face image set includes: obtaining face image samples; extracting upper half face images from the face image samples to generate upper half face image samples; labeling the upper half face image samples with sample tags; and generating an upper half face image set using the labeled upper half face image samples.

[0061] Specifically, the process begins by acquiring an input visible light image sample, detecting faces (with or without masks), aligning it to a preset size A, then dividing the face equally along the vertical line from the higher side of the image sample, taking the upper half of the face, and scaling it back to the preset size A to obtain the upper half face image sample. Sample labels are then used to label the upper half face image sample. Finally, all labeled upper half face image samples are integrated to form an upper half face image set.

[0062] Optionally, the sample label can be a number (e.g., a numeric number, an alphanumeric number, etc.). Labeling upper face image samples can be done by using different numeric numbers to label upper face image samples with different face IDs. For example, suppose there are three upper face image samples with different face IDs: Zhang San's upper face image sample, Li Si's upper face image sample, and Wang Wu's upper face image sample. Then, number 1 can be used to label Zhang San's upper face image sample, number 2 can be used to label Li Si's upper face image sample, and number 3 can be used to label Wang Wu's upper face image sample.

[0063] It should be noted that the number of upper face image samples of the same user can be one or more. When there are multiple upper face image samples of the same user, the same character number is used to label the multiple upper face image samples of the same user.

[0064] S2: Input the upper half face image sample into the first convolutional neural network model to obtain the first image features.

[0065] S3: Input the upper half face image sample into the second convolutional neural network model to obtain the second image features.

[0066] Specifically, feature extraction is performed on the upper half of the face image samples using a first convolutional neural network model and a second convolutional neural network model, respectively, to obtain first image features and second image features. The first image features and second image features have the same dimension.

[0067] S4: Calculate the face recognition classification loss function based on the first image features and sample labels to generate logits; the face recognition classification loss function includes arcface loss function, cosface loss function, ElasticArcFace loss function, ElasticCosFace loss function or GBCosFace loss function.

[0068] S5: Calculate the cross-entropy loss function between logits and the sample labels.

[0069] S6: Calculate the mean square loss function for the first image features and the second image features.

[0070] Specifically, for the first image feature and the corresponding sample label (e.g., label), a face recognition classification loss function (e.g., arcface loss) is first applied to generate logits. Here, logits is a tensor of shape [n, class], where n is the batch size and class is the number of face ID categories, for example, [128, 1000]. The label is also a tensor with dimension n, where n is the batch size. For example, with 128 samples, the label tensor would be [7, 2, 4, 6, 3, 11, ..., 34], where the first component 7 indicates that the label number of the first sample is 7. After obtaining the logits, the cross-entropy loss function (torch.nn.CrossEntropyLoss()) is applied to the logits and label, denoted as loss_s.

[0071] Furthermore, arcface loss is a classic face recognition classification loss, designed to improve intra-class compactness and inter-class separation. Since angular distance has a more direct impact on angles than cosine distance, arcface directly maximizes the classification boundary in the angular space, and its expression is:

[0072]

[0073] The cosface loss function, ElasticArcFace loss function, ElasticCosFace loss function, and GBCosFace loss function are classic face recognition classification losses. In this embodiment, any one of these face recognition classification losses can be selected.

[0074] In addition, the first image features and the second image features are subjected to the mean squared loss function (torch.nn.MSELoss()), denoted as loss_ts.

[0075] S7: Calculate the total loss function based on the cross-entropy loss function and the mean square loss function, perform gradient backpropagation on the total loss function, update the network parameters of the first convolutional neural network model, and continue until the network parameters reach the optimal level to obtain the face recognition model.

[0076] Specifically, the total loss function is loss = a*loss_s + b*loss_ts, where a and b are constants. After calculating the total loss function, gradient inversion is performed on it to update the network parameters of the first convolutional neural network model until the network parameters reach their optimal state, thus completing the training of the first convolutional neural network model and obtaining the face recognition model. Please refer to [link / reference needed] for details. Figure 3As shown, CNN_S represents the first convolutional neural network model, CNN_T represents the second convolutional neural network model, feature_s represents the first image feature, feature_t represents the second image feature, and label represents the sample label.

[0077] In this embodiment, two loss functions, loss_s and loss_ts, are used to learn different upper face categories (i.e., face IDs) distributions, which are compact within classes and large between classes. At the same time, upper face category (i.e., face ID) identity supervision information is also used, i.e., loss_ts is added. This makes the performance of the first convolutional neural network model approach that of the second convolutional neural network model. In other words, it guides the first convolutional neural network model to learn a more optimized feature distribution, thereby improving the recognition accuracy of the face recognition model.

[0078] Furthermore, a specific implementation of the structure of the first convolutional neural network model is provided, and the implementation is described in detail below.

[0079] In one embodiment, the first convolutional neural network model is an improved MobileFaceNet model; wherein the improved MobileFaceNet model includes a first convolutional module, multiple multi-scale channel attention modules, multiple attention-based deep convolutional modules, and a first fully connected layer; the number of multi-scale channel attention modules is the same as the number of attention-based deep convolutional modules, and the first convolutional module, multiple attention-based deep convolutional modules, and the first fully connected layer are connected sequentially; and the multi-scale channel attention modules are positioned between the first convolutional module and the multi-scale channel attention modules, and between the two attention-based deep convolutional modules; the attention-based deep convolutional modules are formed by embedding standardized attention modules into the deep convolutional modules.

[0080] Please refer to Figure 4 The improved MobileFaceNet network model includes a first convolutional module (i.e., Figure 4 (convblock in the middle), multiple multi-scale channel attention modules (i.e.) Figure 4 MS_CAM in the middle), and multiple deep convolutional modules based on attention mechanisms (i.e. Figure 4 The DepthWise-NAM and the first fully connected layer (i.e. Figure 4 (FC in the middle).

[0081] MS_CAM (Multi-scale Channel Attention Module) is a multi-scale channel attention module that addresses the challenges of fusing features at different scales by extracting channel attention through two branches of different scales. MS_CAM primarily continues the ideas of ParseNet, combining local / global features on a CNN and using spatial attention to fuse multiple scales. The structure of MS_CAM is as follows: Figure 5 As shown, MS-CAM addresses channel scale issues through pointwise convolutions, rather than using convolutional kernels of varying sizes. Pointwise convolutions are used to keep MS-CAM as lightweight as possible. MS-CAM does not reside in the backbone network, but rather in the channel attention module, where it incorporates local and global feature context features.

[0082] Attention-based deep convolutional modules (i.e.) Figure 4 The DepthWise-NAM in the algorithm consists of a depthwise convolutional module and a normalization-based attention module (NAM). Specifically, NAM is added to or embedded in the DepthWise layer. As an efficient and lightweight attention mechanism, NAM adopts the module integration of CBAM, including channel and spatial attention sub-modules. For the channel attention sub-module, the scaling factor in batch normalization (BN) is used to measure the variance of the channels and indicate their importance. For the spatial attention sub-module, the scaling factor of BN is applied to the spatial dimension to measure the importance of pixels.

[0083] It should be noted that the number of multi-scale channel attention modules and attention-based deep convolutional modules is the same, and there can be multiple modules, for example, three.

[0084] Next, a specific implementation of the structure of a second convolutional neural network model is given, described in detail below:

[0085] In one embodiment, the second convolutional neural network model is an improved ResNet model; wherein the improved ResNet model includes a second convolutional module, multiple network layers, and a second fully connected layer; each network layer is provided with an attention feature fusion module.

[0086] Specifically, the ResNet network model, also known as a residual neural network, refers to the addition of residual learning to traditional convolutional neural networks. This solves the problems of gradient vanishing and accuracy degradation (training set) in deep networks, allowing the network to become increasingly deeper while maintaining accuracy and controlling speed.

[0087] The ResNet network model typically consists of five convolutional layers, pooling layers, and fully connected layers. The five convolutional layers are designated as conv1_x, conv2_x, conv3_x, conv4_x, and conv5_x. Each of these five convolutional layers contains both regular and residual convolutional modules. The number of regular and residual convolutional modules can vary, resulting in different ResNet network model structures, such as ResNet18, ResNet50, and ResNet101. Furthermore, each convolutional layer can be considered a separate network layer.

[0088] Please refer to Figure 6 The improved ResNet model includes a second convolutional module (i.e., Figure 6 Conv in the network), multiple network layers (i.e. Figure 6 In the second fully connected layer (i.e., layer 1 to layer 4), the second fully connected layer (i.e.) Figure 6 The network layer contains an attention feature fusion module (FC); each network layer has an attention feature fusion module (i.e., FC). Figure 6 (AFF in the middle).

[0089] Among them, AFF, the attention feature fusion module, is suitable for most common scenarios, including feature fusion caused by short and long skip connections and within the Inception layer. Its structure diagram is as follows. Figure 7 As shown.

[0090] Step S130: Compare the image features with the facial features in the face database, and determine whether the face recognition of the user to be identified passes based on the comparison results.

[0091] In one embodiment, comparing image features with facial features in a face database and determining whether the face recognition of the user to be identified passes based on the comparison results includes: calculating the cosine distance between the image features and the facial features to obtain a similarity score; and determining that the face recognition of the user to be identified passes when the similarity score is greater than a preset threshold.

[0092] The preset threshold is a pre-set value, which is usually a range.

[0093] The face recognition method provided in this application first acquires an image of the upper half of the user's face to be recognized; the upper half of the face image is input into a face recognition model to obtain image features; wherein, the face recognition model is obtained by training a first convolutional neural network model based on an upper half of the face image set; during the training of the first convolutional neural network model, the network parameters of the first convolutional neural network model are updated using a first loss function and a second loss function as the total loss function until the model training ends, thus obtaining the face recognition model; the first loss function is the cross-entropy loss function formed by the features of the upper half of the face image samples extracted by the first convolutional neural network model and the sample labels of the upper half of the face image samples; the second loss function is the mean square loss function formed by the different features of the upper half of the face image samples extracted by the first convolutional neural network model and the second convolutional neural network model respectively; the image features are compared with the face features in the face database, and the face recognition of the user to be recognized is determined based on the comparison results.

[0094] This face recognition method uses a face recognition model to extract image features from the upper half of a user's face. The face recognition model is trained on a first convolutional neural network (CNN) model using a set of upper half-face images. During model training, not only the loss function of the first CNN model itself is considered, but also the loss function formed between the first and second CNN models. This is used to update the network parameters of the first CNN model. In other words, the characteristics of both the first and second CNN models are used to train the first CNN, resulting in a face recognition model that possesses characteristics of both models. Furthermore, the second CNN model outperforms the first CNN model, thus improving the accuracy of face recognition. This method can identify users wearing masks as well as those not wearing masks, making it very convenient to use.

[0095] It should be understood that, although Figure 2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated in this document, there is no strict order in which these steps are executed; they can be performed in other orders. Figure 2 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0096] The above-described embodiments of this application disclose a face recognition method. The above-described method can be implemented using various types of devices. Therefore, this application also discloses a face recognition device corresponding to the above method. Specific embodiments are given below for detailed description.

[0097] Please see Figure 8 The present application discloses a face recognition device, which mainly includes:

[0098] The upper half face image acquisition module 810 is used to acquire the upper half face image of the user to be identified.

[0099] The image feature acquisition module 820 is used to input the upper half face image into the face recognition model to obtain image features. The face recognition model is obtained by training a first convolutional neural network model based on the upper half face image set. During the training of the first convolutional neural network model, the network parameters of the first convolutional neural network model are updated using a first loss function and a second loss function as the total loss function until the model training ends, resulting in the face recognition model. The performance of the second convolutional neural network model is superior to that of the first convolutional neural network model. The first loss function is the cross-entropy loss function formed by combining the features of the upper half face image samples extracted by the first convolutional neural network model with the sample labels of the upper half face image samples. The second loss function is the mean square loss function formed by extracting different features of the upper half face image samples using the first and second convolutional neural network models, respectively.

[0100] The comparison module 830 is used to compare image features with facial features in the face database, and determine whether the face recognition of the user to be identified passes based on the comparison results.

[0101] In one embodiment, the apparatus includes:

[0102] The half-face image set acquisition module is used to acquire the upper half-face image set, which includes upper half-face image samples labeled with sample tags.

[0103] The first image feature acquisition module is used to input the upper half face image sample into the first convolutional neural network model to obtain the first image features.

[0104] The second image feature acquisition module is used to input the upper half face image sample into the second convolutional neural network model to obtain the second image features.

[0105] The first calculation module is used to calculate the face recognition classification loss function on the first image features and sample labels, and generate logits; the face recognition classification loss function includes arcface loss function, cosface loss function, ElasticArcFace loss function, ElasticCosFace loss function or GBCosFace loss function.

[0106] The second calculation module is used to calculate the cross-entropy loss function between logits and sample labels.

[0107] The third calculation module is used to calculate the mean square loss function of the first image features and the second image features.

[0108] The face recognition model acquisition module is used to generate a total loss function based on the cross-entropy loss function and the mean square loss function, perform gradient backpropagation on the total loss function, update the network parameters of the first convolutional neural network model, and obtain the face recognition model when the network parameters reach the optimal.

[0109] In one embodiment, the half-face image set acquisition module is used to acquire face image samples, extract the upper half face image from the face image samples to generate upper half face image samples, label the upper half face image samples with sample tags, and generate an upper half face image set using the labeled upper half face image samples.

[0110] In one embodiment, the comparison module 830 is used to calculate the cosine distance between image features and facial features to obtain a similarity score; when the similarity score is greater than a preset threshold, it is determined that the facial recognition of the user to be identified has passed.

[0111] In one embodiment, the first convolutional neural network model and the second convolutional neural network model include a ResNet network model, a VGG network model, a DenseNet network model, a SqueezeNet network model, or a MobileFaceNet network model.

[0112] In one embodiment, the first convolutional neural network model is an improved MobileFaceNet model; wherein the improved MobileFaceNet model includes a first convolutional module, multiple multi-scale channel attention modules, multiple attention-based deep convolutional modules, and a first fully connected layer; the number of multi-scale channel attention modules is the same as the number of attention-based deep convolutional modules, and the first convolutional module, multiple attention-based deep convolutional modules, and the first fully connected layer are connected sequentially; and the multi-scale channel attention modules are positioned between the first convolutional module and the multi-scale channel attention modules, and between the two attention-based deep convolutional modules; the attention-based deep convolutional modules are formed by embedding standardized attention modules into the deep convolutional modules.

[0113] In one embodiment, the second convolutional neural network model is an improved ResNet model; wherein the improved ResNet model includes a second convolutional module, multiple network layers, and a second fully connected layer; each network layer is provided with an attention feature fusion module.

[0114] Specific limitations regarding the facial recognition device can be found in the method limitations section above, and will not be repeated here. Each module in the aforementioned device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the terminal device, or stored in software in the memory of the terminal device, so that the processor can call and execute the corresponding operations of each module.

[0115] Please refer to Figure 9 , Figure 9 This illustration shows a structural block diagram of a terminal device provided in an embodiment of this application. The terminal device 90 can be a computer device. The terminal device 90 in this application may include one or more of the following components: a processor 92, a memory 94, and one or more application programs, wherein the one or more application programs can be stored in the memory 94 and configured to be executed by the one or more processors 92, and the one or more application programs are configured to perform the methods described in the above-described face recognition method embodiments.

[0116] The processor 92 may include one or more processing cores. The processor 92 connects to various parts within the terminal device 90 using various interfaces and lines, and performs various functions and processes data of the terminal device 90 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 94, and by calling data stored in the memory 94. Optionally, the processor 92 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 92 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 92 and may be implemented separately using a communication chip.

[0117] The memory 94 may include random access memory (RAM) or read-only memory (ROM). The memory 94 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 94 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the terminal device 90 during use.

[0118] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the terminal device to which the solution of this application is applied. A specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0119] In summary, the terminal device provided in this application embodiment is used to implement the corresponding face recognition method in the foregoing method embodiment and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0120] Please see Figure 10 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable storage medium 1000 stores program code, which can be called by a processor to execute the method described in the above-described face recognition method embodiment.

[0121] The computer-readable storage medium 1000 may be an electronic storage device such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 1000 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 1000 has storage space for program code 1002 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 1002 may be compressed, for example, in a suitable form.

[0122] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0123] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A face recognition method, characterized in that, The method includes: Obtain an image of the upper half of the user's face to be identified; The upper half of the face image is input into a face recognition model to obtain image features; The face recognition model is obtained by training a first convolutional neural network model based on a set of upper face images. During the training of the first convolutional neural network model, the network parameters are updated using a first loss function and a second loss function as the total loss function until the model training is complete, resulting in the face recognition model. The second convolutional neural network model outperforms the first convolutional neural network model. The first convolutional neural network model is an improved MobileFaceNet model. The improved MobileFaceNet model includes a first convolutional module, multiple multi-scale channel attention modules, multiple attention-based deep convolutional modules, and a first fully connected layer. The number of multi-scale channel attention modules is the same as the number of attention-based deep convolutional modules. The first convolutional module, the multiple attention-based deep convolutional modules, and the first fully connected layer are sequentially connected. The multi-scale channel attention modules are positioned between the first convolutional module and the multi-scale channel attention modules, and between two attention-based deep convolutional modules. The attention-based deep convolutional modules are formed by embedding standardized attention modules into the deep convolutional modules. The first loss function is the cross-entropy loss function, which is formed by combining the features of the upper half face image samples extracted from the upper half face image set using the first convolutional neural network model with the sample labels of the upper half face image samples. The second loss function is the mean square loss function, which is formed by extracting different features of the upper half face image samples using the first convolutional neural network model and the second convolutional neural network model, respectively. During model training, the network parameters of the first convolutional neural network model are updated, but the network parameters of the second convolutional neural network model are not updated. The cross-entropy loss function and the mean square loss function are used to learn different upper half face category distributions, and the addition of the mean square loss function makes the performance of the first convolutional neural network model approach the performance of the second convolutional neural network model. The image features are compared with facial features in the face database, and the facial recognition of the user to be identified is determined based on the comparison results.

2. The method according to claim 1, characterized in that, The face recognition model is obtained through the following method: Obtain an upper half face image set, the upper half face image set including upper half face image samples labeled with sample tags; The upper half face image sample is input into the first convolutional neural network model to obtain the first image feature; The upper half face image sample is input into the second convolutional neural network model to obtain the second image features; The face recognition classification loss function is calculated on the first image features and the sample labels to generate logits; the face recognition classification loss function includes arcface loss function, cosface loss function, ElasticArcFace loss function, ElasticCosFace loss function or GBCosFace loss function; Calculate the cross-entropy loss function between the logits and the sample labels; Calculate the mean square loss function for the first image feature and the second image feature; A total loss function is generated based on the cross-entropy loss function and the mean square loss function. Gradient backpropagation is performed on the total loss function to update the network parameters of the first convolutional neural network model until the network parameters reach the optimal state, so as to obtain the face recognition model.

3. The method according to claim 2, characterized in that, The acquisition of the upper half face image set includes: Obtain a face image sample, and extract the upper half of the face image from the face image sample to generate an upper half face image sample; The upper half face image samples are labeled with sample tags, and the labeled upper half face image samples are used to generate the upper half face image set.

4. The method according to claim 1, characterized in that, The image features are compared with facial features in a face database, and the face recognition of the user to be identified is determined based on the comparison results, including: Calculate the cosine distance between the image features and the face features to obtain a similarity score; When the similarity score is greater than a preset threshold, the face recognition of the user to be identified is deemed successful.

5. The method according to any one of claims 1-4, characterized in that, The first convolutional neural network model and the second convolutional neural network model include ResNet network model, VGG network model, DenseNet network model, SqueezeNet network model or MobileFaceNet network model.

6. The method according to any one of claims 1-4, characterized in that, The second convolutional neural network model is an improved ResNet model; wherein the improved ResNet model includes a second convolutional module, multiple network layers, and a second fully connected layer; Each of the network layers is equipped with an attention feature fusion module.

7. A face recognition device, characterized in that, The device includes: The upper half face image acquisition module is used to acquire the upper half face image of the user to be identified; The image feature acquisition module is used to input the upper half face image into the face recognition model to obtain image features; The face recognition model is obtained by training a first convolutional neural network model based on a set of upper face images. During the training of the first convolutional neural network model, the network parameters are updated using a first loss function and a second loss function as the total loss function until the model training is complete, resulting in the face recognition model. The second convolutional neural network model outperforms the first convolutional neural network model. The first convolutional neural network model is an improved MobileFaceNet model. The improved MobileFaceNet model includes a first convolutional module, multiple multi-scale channel attention modules, multiple attention-based deep convolutional modules, and a first fully connected layer. The number of multi-scale channel attention modules is the same as the number of attention-based deep convolutional modules. The first convolutional module, the multiple attention-based deep convolutional modules, and the first fully connected layer are sequentially connected. The multi-scale channel attention modules are positioned between the first convolutional module and the multi-scale channel attention modules, and between two attention-based deep convolutional modules. The attention-based deep convolutional modules are formed by embedding standardized attention modules into the deep convolutional modules. The first loss function is the cross-entropy loss function, which is formed by combining the features of the upper half face image samples extracted from the upper half face image set using the first convolutional neural network model with the sample labels of the upper half face image samples. The second loss function is the mean square loss function, which is formed by extracting different features of the upper half face image samples using the first convolutional neural network model and the second convolutional neural network model, respectively. During model training, the network parameters of the first convolutional neural network model are updated, but the network parameters of the second convolutional neural network model are not updated. The cross-entropy loss function and the mean square loss function are used to learn different upper half face category distributions, and the addition of the mean square loss function makes the performance of the first convolutional neural network model approach the performance of the second convolutional neural network model. The comparison module is used to compare the image features with the facial features in the face database, and determine whether the face recognition of the user to be identified passes based on the comparison results.

8. A terminal device, characterized in that, include: Memory; One or more processors are coupled to the memory; One or more applications, wherein the one or more applications are stored in memory and configured to be executed by one or more processors, and the one or more applications are configured to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1-6.

Images