Face recognition method and device, equipment, medium and product

Abstract

The embodiment of the invention provides a face recognition method and device, equipment, a medium and a product, and the method comprises the steps: obtaining N first images, N being a positive integer; recognizing each first image by adopting a first recognition model to obtain a recognition result of each first image; the recognition result of the first image comprises a second image containing a face region; m third images are screened out from the N second images, and M is a positive integer smaller than or equal to N; recognizing each third image by adopting a second recognition model to obtain a target face feature vector corresponding to each third image; and matching the target face feature vectors based on a preset corresponding relationship between face feature vectors and user identity information, and identifying target user identity information corresponding to each target face feature vector. According to the embodiment of the invention, the accuracy of face recognition is improved.

Description

Technical Field

[0001] The present application belongs to the field of face recognition technology, and in particular, relates to a method, device, equipment, medium and product for face recognition. Background Art

[0002] With the rapid development of science and technology, image processing and computer recognition technology, especially face recognition technology, has penetrated into all aspects of real life, especially in the fields of security monitoring, intelligent security, intelligent transportation, etc., playing a vital role.

[0003] However, in the prior art, a facial image is generally acquired by an image acquisition device, and a traditional face recognition algorithm is used to recognize the facial image of the image acquisition device. However, the accuracy of face recognition by this method is low. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device, medium and product for face recognition, which improve the accuracy of face recognition.

[0005] In a first aspect, an embodiment of the present application provides a method for face recognition, the method comprising:

[0006] Obtain N first images, where N is a positive integer;

[0007] Using the first recognition model to recognize each first image, and obtaining a recognition result of each first image; the recognition result of the first image includes a second image containing a face area;

[0008] Selecting M third images from the N second images, where M is a positive integer less than or equal to N;

[0009] Using the second recognition model to recognize each third image, and obtaining a target face feature vector corresponding to each third image;

[0010] Based on the preset correspondence between facial feature vectors and user identity information, the target facial feature vectors are matched to identify the target user identity information corresponding to each target facial feature vector.

[0011] In an optional implementation manner of the first aspect, screening out M third images from N second images includes:

[0012] Based on at least one of face confidence, face posture information and image quality information, M third images that meet preset conditions are screened out from N second images; the face confidence is used to characterize the probability that the second image contains a face area.

[0013] In an optional implementation of the first aspect, the image quality information includes at least one of image clarity, image brightness, and image contrast;

[0014] In the case where the image quality information includes image clarity, the preset condition is that the image clarity is greater than a preset clarity threshold; or,

[0015] In the case where the image quality information includes image brightness, the preset condition is that the image brightness is within a preset brightness range; or,

[0016] In the case where the image quality information includes image contrast, the preset condition is that the image contrast is within a preset contrast range.

[0017] In an optional implementation manner of the first aspect, the preset condition includes that the face confidence is greater than a preset confidence threshold; or,

[0018] The face posture information includes at least one of a yaw angle, a pitch angle, and a roll angle;

[0019] When the face posture information includes a yaw angle, the preset condition is that the yaw angle is within a first angle range; or,

[0020] When the face posture information includes a pitch angle, the preset condition is that the pitch angle is within a second angle range; or,

[0021] When the facial posture information includes a roll angle, the preset condition is that the roll angle is within a third angle range.

[0022] In an optional implementation of the first aspect, the preset condition is that the face confidence is greater than a preset confidence threshold.

[0023] In an optional implementation of the first aspect, the facial posture information includes at least one of a yaw angle, a pitch angle, and a roll angle;

[0024] When the face posture information includes a yaw angle, the preset condition is that the yaw angle is within a first angle range; or,

[0025] When the face posture information includes a pitch angle, the preset condition is that the pitch angle is within a second angle range; or,

[0026] When the facial posture information includes a roll angle, the preset condition is that the roll angle is within a third angle range.

[0027] In an optional implementation of the first aspect, the recognition result of the first image further includes key point positions of a plurality of facial key points in the face region;

[0028] Before selecting M third images meeting a preset condition from the N second images based on at least one of the face confidence, the face posture information, and the image quality information, the method further includes:

[0029] For each of the N second images, the face region is corrected based on key point positions of a plurality of face key points in the face region contained in the second image to obtain N corrected second images.

[0030] In an optional implementation of the first aspect, before using the second recognition model to recognize each third image to obtain a target face feature vector corresponding to each third image, the method further includes:

[0031] obtaining an image resolution of each third image;

[0032] When the image resolution of the third image is less than the preset resolution threshold, the image resolution of the third image is adjusted until the image resolution of the adjusted third image is greater than or equal to the preset resolution threshold, thereby obtaining the adjusted third image.

[0033] In an optional implementation manner of the first aspect, the preset correspondence relationship between the facial feature vector and the user identity information includes a plurality of user identity information and a facial feature vector corresponding to each user identity information;

[0034] The facial feature vector corresponding to each user identity information includes a first sub-feature vector and a second sub-feature vector. The first sub-feature vector is determined based on the obstructed facial image corresponding to the user identity information, and the second sub-feature vector is determined based on the unobstructed facial image corresponding to the user identity information.

[0035] In a second aspect, an embodiment of the present application provides a face recognition device, the device comprising:

[0036] An acquisition module, used for acquiring N first images, where N is a positive integer;

[0037] A recognition module, used for recognizing each first image using a first recognition model to obtain a recognition result of each first image; the recognition result of the first image includes a second image containing a face area;

[0038] A screening module, used to screen out M third images from the N second images, where M is a positive integer less than or equal to N;

[0039] The recognition module is further used to recognize each third image using the second recognition model to obtain a target face feature vector corresponding to each third image;

[0040] The matching module is used to match the target facial feature vectors based on the preset correspondence between the facial feature vectors and the user identity information, and identify the target user identity information corresponding to each target facial feature vector.

[0041] In a third aspect, an electronic device is provided, comprising: a memory for storing computer program instructions; and a processor for reading and running the computer program instructions stored in the memory to execute the face recognition method provided by any optional implementation in the first aspect.

[0042] In a fourth aspect, a computer storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method for face recognition provided by any optional implementation in the first aspect is implemented.

[0043] In a fifth aspect, a computer program product is provided, the computer program product comprising a computer program, and when the computer program is executed by a processor, the method for face recognition provided by any optional implementation in the first aspect is implemented.

[0044] In the embodiment of the present application, N first images can be obtained, and each first image can be preliminarily recognized using the first recognition model to obtain the recognition result of each first image, and the recognition result of the first image includes a second image containing a face area, and then M third images with higher quality can be screened out from the N second images, and each third image can be further recognized using the second recognition model to obtain the target face feature vector corresponding to each third image, and then each target face feature vector can be matched based on the correspondence between the preset face feature vector and the user identity information to accurately identify the target user identity information corresponding to each target face feature vector. In this way, by using the first recognition model to perform initial recognition on the image, a face image containing a face area is obtained, and then the second model is used to perform face recognition again on the image containing the face area, thereby improving the face recognition accuracy when the face area is small, and in this process, by screening some images from the acquired multiple images for final face recognition, not only the face recognition accuracy is further improved, but also the resource consumption is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0046] Figure 1 This is one of the flowcharts of a face recognition method provided in an embodiment of the present application;

[0047] Figure 2 This is a second flow chart of a face recognition method provided in an embodiment of the present application;

[0048] Figure 3 It is a flowchart of a training method for a first recognition model provided in an embodiment of the present application;

[0049] Figure 4 It is a flowchart of a training method for a second recognition model provided in an embodiment of the present application;

[0050] Figure 5 This is a third flow chart of a face recognition method provided in an embodiment of the present application;

[0051] Figure 6 It is a structural schematic diagram of a face recognition device provided in an embodiment of the present application;

[0052] Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0054] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0055] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0056] As mentioned in the background technology, in the prior art, facial images are generally collected by image acquisition devices, and the facial images are recognized using traditional facial recognition algorithms, such as RetinaFace and other facial detection algorithms. However, the facial recognition effect of this algorithm is not good, and Retinace is a relatively heavy algorithm that consumes more resources and is not suitable for edge devices with low computing power. In addition, when the face is small when photographed by a high-position camera, the accuracy of facial recognition is low.

[0057] Based on this, in order to solve the problem of low accuracy of face recognition in the prior art, the embodiment of the present application provides a method, device, equipment, medium and product for face recognition, which can obtain N first images, and use the first recognition model to perform preliminary recognition on each first image to obtain the recognition result of each first image, and the recognition result of the first image includes the second image containing the face area, and then M third images with higher quality can be screened out from the N second images, and each third image can be further recognized by the second recognition model to obtain the target face feature vector corresponding to each third image, and then each target face feature vector can be matched based on the correspondence between the preset face feature vector and the user identity information to accurately identify the target user identity information corresponding to each target face feature vector. In this way, by using the first recognition model to perform initial recognition on the image, a face image containing the face area is obtained, and then the second model is used to perform face recognition again on the image containing the face area, which improves the accuracy of face recognition when the face area is small, and in this process, by screening some images from the acquired multiple images for final face recognition, not only the accuracy of face recognition is further improved, but also resource consumption is reduced.

[0058] It should be noted that the execution subject of the face recognition method provided in the embodiment of the present application can be a face recognition device, or a control module in the face recognition device for executing the face recognition method. In the embodiment of the present application, the face recognition method provided in the embodiment of the present application is described by taking the face recognition device executing the face recognition method as an example.

[0059] The face recognition method provided in the embodiment of the present application is described in detail below through specific embodiments in conjunction with the accompanying drawings.

[0060] Figure 1 It is a flowchart of a face recognition method provided in an embodiment of the present application.

[0061] like Figure 1 As shown, the execution subject of the method may be a face recognition device, based on which, the method may specifically include the following steps:

[0062] S110, acquiring N first images.

[0063] Each of the N first images may be a face image containing a face region, and the face image may be an occluded face image or an unoccluded face image, which is not specifically limited here. In addition, N is a positive integer.

[0064] In one embodiment, the above S110 may specifically include the following steps:

[0065] Acquire video data acquired by the image acquisition device within a first time, where the video data includes N fourth images;

[0066] Determine N fourth images, which are the N first images.

[0067] Specifically, the face recognition device can obtain video data of the image acquisition device within the first time, and the video data may include N fourth images. Based on this, the face recognition device can directly obtain N first images by determining the N fourth images as N first images.

[0068] The image acquisition device may be a related device for acquiring images, such as a monitoring device or a related device for acquiring facial images in an access control system, which is not specifically limited here. The fourth image may be a facial image containing a facial area.

[0069] In addition, the first time mentioned above may be pre-set based on actual experience or circumstances. For example, the first time may be 1 minute or 10 seconds, which is not specifically limited here.

[0070] In this embodiment, N first images can be acquired by determining that N fourth images acquired by the image acquisition device within the first time are N first images, thereby improving the convenience of acquiring the first images.

[0071] In order to accurately acquire N first images, in one embodiment, the above S110 may specifically include the following steps:

[0072] Acquire video data acquired by the image acquisition device within a first time, where the video data includes N fourth images;

[0073] For each of the N fourth images, the fourth image is preprocessed according to a preset preprocessing method to obtain N first images.

[0074] The preset preprocessing method may be preset based on actual experience or circumstances, for example, the preset preprocessing method includes but is not limited to denoising the image, scaling the image according to a preset size, etc., which are not specifically limited here. The above-mentioned preset size may be based on actual experience or circumstances, which are not specifically limited here.

[0075] Specifically, the face recognition device can obtain video data captured by the image acquisition device in the first time, the video data includes N fourth images, and preprocess each of the N fourth images according to a preset preprocessing method to obtain N first images.

[0076] In this embodiment, each fourth image included in the acquired video data can be preprocessed to obtain a more accurate first image, thereby facilitating subsequent improvement in the accuracy of face recognition.

[0077] S120: Recognize each first image using a first recognition model to obtain a recognition result of each first image.

[0078] In some embodiments, the recognition result of the first image may include a second image including a face region, wherein the image size of the second image may be smaller than or equal to the size of the first image.

[0079] In addition, the first recognition model may be a related model for preliminarily recognizing the face area in the first image. For example, the first recognition model may be a YOLO8 model, which is not specifically limited here.

[0080] Specifically, after obtaining N first images, the face recognition device can use the first recognition model to perform preliminary recognition on each of the N first images to obtain a recognition result of each first image.

[0081] S130, selecting M third images from the N second images.

[0082] Wherein, M is a positive integer less than or equal to N.

[0083] After each first image is recognized using the first recognition model to obtain the recognition result of each first image, and the recognition result of each first image includes a second image containing a face area, the face recognition device can obtain N second images, and then can screen out M third images from the N second images for subsequent recognition.

[0084] S140, using the second recognition model to recognize each third image to obtain a target face feature vector corresponding to each third image.

[0085] Among them, the above-mentioned second recognition model can be a related model for further face recognition of the third image. For example, the second recognition model can be a multi-task convolutional neural network (Multi-task Cascaded Convolutional Neural Networks, MT-CNN), which is not specifically limited here.

[0086] Specifically, after selecting M third images from N second images, the face recognition device can further identify each of the M third images using the second recognition model to determine the target face feature vector corresponding to the third image.

[0087] S150, based on the preset correspondence between the facial feature vector and the user identity information, each target facial feature vector is matched to identify the target user identity information corresponding to the target facial feature vector.

[0088] The corresponding relationship between the preset facial feature vector and the user identity information may be preset based on actual experience or circumstances.

[0089] Specifically, after obtaining the target facial feature vector corresponding to each third image, the face recognition device can match the target facial feature vector based on the preset correspondence between the facial feature vector and the user identity information, and identify the target user identity information corresponding to the target facial feature vector.

[0090] In the embodiment of the present application, N first images can be obtained, and each first image can be preliminarily recognized using the first recognition model to obtain the recognition result of each first image, and the recognition result of the first image includes a second image containing a face area, and then M third images with higher quality can be screened out from the N second images, and each third image can be further recognized using the second recognition model to obtain the target face feature vector corresponding to each third image, and then each target face feature vector can be matched based on the correspondence between the preset face feature vector and the user identity information to accurately identify the target user identity information corresponding to each target face feature vector. In this way, by using the first recognition model to perform initial recognition on the image, a face image containing a face area is obtained, and then the second model is used to perform face recognition again on the image containing the face area, thereby improving the face recognition accuracy when the face area is small, and in this process, by screening some images from the acquired multiple images for final face recognition, not only the face recognition accuracy is further improved, but also the resource consumption is reduced.

[0091] Since the N second images are obtained by respectively recognizing the N first images, and the facial areas included in each of the N first images may be for the same user, therefore, if each of the N second images is recognized, it will cause unnecessary waste of resources.

[0092] The face recognition method provided in the embodiment of the present application takes into account that the face areas included in the acquired N second images may be the face of the same user. Based on this, in order to avoid unnecessary waste of resources, it is necessary to screen out M third images from the N second images for subsequent recognition. Thus, in order to accurately screen out the M third images from the N second images, in one embodiment, the above S130 may specifically include the following steps:

[0093] Based on at least one of the face confidence, the face posture information, and the image quality information, M third images meeting a preset condition are screened out from the N second images.

[0094] The face confidence is used to characterize the probability that the second image contains a face area. Specifically, the greater the face confidence, the greater the probability that the second image contains a face area, and vice versa. The above-mentioned face posture information can be relevant posture information of the face relative to a reference coordinate system (such as a camera coordinate space). The above-mentioned image quality information can be relevant information for describing and evaluating the visual characteristics of the image.

[0095] In addition, the above-mentioned preset conditions can be preset based on actual experience or circumstances, and are not specifically limited here.

[0096] Specifically, after obtaining N second images, the face recognition device can screen out M third images that meet preset conditions from the N first images based on at least one of face confidence, face posture information and image quality information.

[0097] In this embodiment, based on at least one of the face confidence, face posture information and image quality information, a better face image can be screened out from N second images, so that face recognition can be further performed on the screened images. This not only improves the accuracy of face recognition, but also avoids unnecessary waste of resources.

[0098] Based on the above content, it can be known that the face recognition method provided in the embodiment of the present application can screen out M third images that meet the preset conditions from N second images based on at least one of face confidence, face posture information, and image quality information. Thus, in order to accurately and in detail describe the face recognition method provided in the embodiment of the present application, in one embodiment, the above-mentioned image quality information may include at least one of image clarity, image brightness, and image contrast.

[0099] In the case where the image quality information includes image clarity, the preset condition is that the image clarity is greater than a preset clarity threshold; or,

[0100] In the case where the image quality information includes image brightness, the preset condition is that the image brightness is within a preset brightness range; or,

[0101] In the case where the image quality information includes image contrast, the preset condition is that the image contrast is within a preset contrast range.

[0102] Among them, the preset clarity threshold may be a clarity threshold preset based on actual experience or situation. The above-mentioned preset brightness range may be a brightness range preset based on actual experience or situation. The above-mentioned preset contrast range may be a contrast range preset based on actual experience or situation. The embodiments of the present application do not specifically limit the above-mentioned preset clarity threshold, preset brightness range, and preset contrast range.

[0103] In this way, the face recognition device can select M third images that meet the preset conditions from N second images based only on the image quality information. In this process, since the image quality information can include at least one of image clarity, image brightness, and image contrast, the above preset conditions can be determined based on the specific information included in the image quality information.

[0104] In one example, when the image quality information includes image clarity, the preset condition is that the image clarity is greater than a preset clarity threshold. In this way, M third images whose image clarity is greater than the preset clarity threshold can be screened out from N second images based on the image quality information.

[0105] In another example, when the image quality information includes image brightness, the preset condition may be that the image brightness is within a preset brightness range. In this way, M third images whose image brightness is within the preset brightness range can be screened out from N second images based on the image quality information.

[0106] In another example, when the image quality information includes image contrast, the preset condition may be that the image contrast is within a preset contrast range. In this way, M third images whose image contrast is within the preset contrast range can be screened out from N second images based on the image quality information.

[0107] In this embodiment, M third images that meet the requirements can be screened out from N second images based on the image quality information, which not only avoids unnecessary waste of resources but also facilitates subsequent accurate face recognition, thereby improving the accuracy of face recognition.

[0108] Based on this, in order to further improve the accuracy of face recognition, in one embodiment, the preset condition includes that the face confidence is greater than a preset confidence threshold; or,

[0109] The face posture information includes at least one of a yaw angle, a pitch angle, and a roll angle;

[0110] When the face posture information includes a yaw angle, the preset condition is that the yaw angle is within a first angle range; or,

[0111] When the face posture information includes a pitch angle, the preset condition is that the pitch angle is within a second angle range; or,

[0112] When the facial posture information includes a roll angle, the preset condition is that the roll angle is within a third angle range.

[0113] Among them, the above-mentioned preset confidence threshold may be a confidence threshold preset based on actual experience or situation. The above-mentioned first angle range may be a yaw angle range preset based on actual experience or situation, the above-mentioned second angle range may be a pitch angle range preset based on actual experience or situation, and the above-mentioned third angle range may be a roll angle range preset based on actual experience or situation. The embodiments of the present application do not specifically limit the above-mentioned preset confidence threshold, the first angle range, the second angle range, and the third angle range.

[0114] In addition, it should be noted that the first angle range, the second angle range and the third angle range may be the same or different, and are not specifically limited here. Exemplarily, the first angle range, the second angle range and the third angle range may all be [-15°, +15°].

[0115] In this way, the face recognition device can screen the N second images based on the image quality information, and can further screen the N second images based on at least one of the face confidence and face posture information, so as to screen out M third images that meet the preset conditions.

[0116] In one example, after obtaining N second images, the face recognition device can, based on the image quality information and the face confidence, filter out M third images from the N second images, whose image quality information meets the preset conditions corresponding to the image quality and whose face confidence is greater than a preset confidence threshold.

[0117] In another example, after obtaining N second images, the face recognition device can, based on the image quality information and the face posture information, screen out M third images from the N second images, wherein the image quality information of the second images meets the preset conditions corresponding to the image quality, and the face posture information corresponding to the second images meets the preset conditions corresponding to the face posture information.

[0118] In another example, after obtaining N second images, the face recognition device can screen out M third images from the N second images based on the image quality information, the face confidence and the face posture information, wherein the image quality information of the second image meets the preset condition corresponding to the image quality, the face confidence of the second image is greater than the preset confidence threshold, and the face posture information corresponding to the second image meets the preset condition corresponding to the face posture information.

[0119] It should be noted that, since the above-mentioned facial posture information may include at least one of the yaw angle, the pitch angle, and the roll angle, in the process of selecting N second images based on the facial posture information to obtain M third images, the preset condition corresponding to the facial posture information needs to be determined according to the specific information included in the facial posture information. For example, in the case where the facial posture information includes the yaw angle, the preset condition may be that the yaw angle is within the first angle range, so that the second image with the yaw angle within the first angle range can be selected from the N second images as the third image. For another example, in the case where the facial posture information includes the pitch angle, the preset condition may be that the pitch angle is within the second angle range, so that the second image with the pitch angle within the second angle range can be selected from the N second images and used as the third image. Alternatively, in the case where the facial posture information includes the roll angle, the preset condition may be that the roll angle is within the third angle range, so that the second image with the roll angle within the third angle range can be selected from the N second images and used as the third image.

[0120] It should be noted that the embodiment of the present application does not specifically limit the specific execution order of the face recognition device for determining whether the second image meets the preset conditions.

[0121] In this embodiment, in the process of selecting M third images that meet the preset conditions from N second images based on the image quality information, the N second images can be further selected based on at least one of the face confidence and the face posture information to select the M third images that meet the preset conditions. In this way, it is convenient to accurately perform face recognition in the subsequent process, and the accuracy of face recognition is further improved.

[0122] In another embodiment, the preset condition may be that the face confidence is greater than a preset confidence.

[0123] The preset confidence threshold may be a confidence threshold preset based on actual experience or circumstances, and is not specifically limited here.

[0124] In this way, after obtaining N second images, the face recognition device can screen out M third images whose face confidences are greater than a preset confidence threshold from the N second images based on the face confidences.

[0125] In this embodiment, based on the face confidence, M third images meeting the recognition requirements can be screened out from the N second images for use in the next step of face recognition, thereby improving the accuracy of face recognition.

[0126] In yet another embodiment, the facial posture information includes at least one of a yaw angle, a pitch angle, and a roll angle;

[0127] When the face posture information includes a yaw angle, the preset condition is that the yaw angle is within a first angle range; or,

[0128] When the face posture information includes a pitch angle, the preset condition is that the pitch angle is within a second angle range; or,

[0129] When the facial posture information includes a roll angle, the preset condition is that the roll angle is within a third angle range.

[0130] In this way, after obtaining the N second images, the face recognition device can screen out M third images whose face posture information meets the preset conditions from the N second images based on the face posture information.

[0131] Alternatively, after obtaining N second images, the face recognition device can, based on face confidence and face posture information, screen out M third images from the N second images whose face confidence is greater than a preset confidence threshold and whose face posture information meets preset conditions.

[0132] It should be noted that, since the above-mentioned facial posture information may include at least one of the yaw angle, the pitch angle, and the roll angle, in the process of selecting N second images based on the facial posture information to obtain M third images, the preset condition corresponding to the facial posture information needs to be determined according to the specific information included in the facial posture information. For example, in the case where the facial posture information includes the yaw angle, the preset condition may be that the yaw angle is within the first angle range, so that the second image with the yaw angle within the first angle range can be selected from the N second images as the third image. For another example, in the case where the facial posture information includes the pitch angle, the preset condition may be that the pitch angle is within the second angle range, so that the second image with the pitch angle within the second angle range can be selected from the N second images and used as the third image. Alternatively, in the case where the facial posture information includes the roll angle, the preset condition may be that the roll angle is within the third angle range, so that the second image with the roll angle within the third angle range can be selected from the N second images and used as the third image.

[0133] In this embodiment, M third images whose facial posture information meets the requirements can be screened out from N second images, or M third images whose facial confidence and facial posture information meet the requirements can be screened out from N second images for further face recognition, thereby improving the accuracy of face recognition.

[0134] In addition, it should be noted that the face recognition method provided in the embodiment of the present application needs to obtain at least one of the face confidence, face posture information and image quality information of each second image before screening out M third images that meet preset conditions from N first images based on at least one of the face confidence, face posture information and image quality information.

[0135] Based on this, with respect to the face confidence, since the recognition result of the first image obtained by the first recognition model recognizing the first image also includes the face confidence, the face recognition device can directly obtain the face confidence.

[0136] Regarding image quality information, the face recognition device can, after obtaining N second images, use relevant technologies to process each second image to obtain image quality information of each second image. The embodiment of the present application does not specifically limit the relevant technology used to obtain the image quality information of the second image.

[0137] Regarding facial posture information, since the recognition result of the first image obtained by the first recognition model for recognizing the first image also includes the key point positions of multiple facial key points in the face area, the face recognition device can determine the facial posture information of the second image based on the key point positions of multiple facial key points in the face area contained in each second image.

[0138] Thus, in some embodiments, the facial posture information may include at least one of a yaw angle, a pitch angle, and a roll angle, and the multiple facial key points may include at least a left eye key point, a right eye key point, a left corner of the mouth key point, a right corner of the mouth key point, and a nose key point. Based on this, the face recognition method provided in the embodiment of the present application may also include the following steps:

[0139] For each of the N second images, based on the key point positions of the multiple facial key points, determining a first distance between a left eye key point and a right eye key point in the horizontal direction, and determining a second distance between a left mouth corner key point and a right mouth corner key point in the horizontal direction;

[0140] Based on the first mapping relationship, a first ratio between the first distance and the second distance is matched to obtain a yaw angle; and / or,

[0141] For each of the N second images, based on the key point positions of the multiple facial key points, determining a third distance between the first key point and the nose key point in the vertical direction, and a fourth distance between the second key point and the nose key point in the vertical direction, wherein the first key point includes a left eye key point or a right eye key point, and the second key point includes a left mouth corner key point or a right mouth corner key point;

[0142] Based on the second mapping relationship, matching the second ratio between the third distance and the fourth distance to obtain the pitch angle; and / or,

[0143] For each of the N second images, determining a fifth distance between the left eye key point and the right eye key point in the vertical direction based on key point positions of the plurality of facial key points;

[0144] The five distances are matched based on the third mapping relationship to obtain the roll angle.

[0145] The first mapping relationship, the second mapping relationship and the third mapping relationship may all be mapping relationships preset based on actual experience or circumstances. Specifically, the first mapping relationship may include a mapping relationship between the first ratio and the yaw angle, the second mapping relationship may include a mapping relationship between the second ratio and the pitch angle, and the third mapping relationship may include a mapping relationship between the fifth distance and the roll angle.

[0146] Since the facial posture information may include at least one of the yaw angle, the pitch angle and the roll angle, and the above-mentioned multiple facial key points include at least the left eye key point, the right eye key point, the left corner of the mouth key point, the right corner of the mouth key point, and the nose key point. Based on this, the face recognition device can, for each of the N second images, based on the key point positions of the multiple facial key points, first determine the first distance in the horizontal direction between the left eye key point and the right eye key point, and the second distance in the horizontal direction between the left corner of the mouth key point and the right corner of the mouth key point, and then match the first ratio between the first distance and the second distance based on the first mapping relationship to obtain the yaw angle corresponding to the first ratio between the first distance and the second distance. And / or,

[0147] The face recognition device can also determine, for each of the N second images, a third distance between the first key point and the nose key point in the vertical direction, and a fourth distance between the second key point and the nose key point in the vertical direction based on the key point positions of the multiple face key points, and then match the second ratio between the third distance and the fourth distance based on the second mapping relationship to obtain a pitch angle corresponding to the second ratio between the third distance and the fourth distance. And / or,

[0148] The face recognition device can also determine, for each of the N second images, the fifth distance between the left eye key point and the right eye key point in the vertical direction based on the key point positions of multiple facial key points, and then match the fifth distance based on the third mapping relationship to determine the roll angle.

[0149] In this embodiment, the positions of multiple facial key points in the face area of ​​each second image can be used to determine facial posture information, so that images that meet the requirements can be screened based on the facial posture information for further facial recognition, thereby improving the accuracy of facial recognition.

[0150] Taking into account that in actual applications, the facial images captured by the image acquisition device are not taken when the face is completely facing the camera, this may lead to lower accuracy in subsequent face recognition. Based on this, in order to perform subsequent face recognition more accurately, in one embodiment, the recognition result of the above-mentioned first image may also include the key point positions of multiple facial key points in the face area.

[0151] Thus, before selecting M third images meeting preset conditions from N second images based on at least one of face confidence, face posture information, and image quality information, the above-mentioned face recognition method may further include the following steps:

[0152] For each of the N second images, the face region is corrected based on key point positions of a plurality of face key points in the face region contained in the second image to obtain N corrected second images.

[0153] Specifically, the face recognition device can correct the face area of ​​each of the N second images based on the key point positions of multiple face key points in the face area contained in the second image to obtain N corrected second images.

[0154] In this embodiment, after obtaining a second image including a face area, a key point alignment operation can be performed on the face area based on the key point positions of multiple face key points in the face area to correct the facial posture of the face, thereby avoiding the low accuracy of face recognition due to the face posture not meeting the requirements, and facilitating more accurate capture of subtle feature changes of the face in the subsequent face recognition process.

[0155] In addition, in practical applications, due to the high position of the image acquisition device, the size of the face area in the acquired image may be small, so the face resolution in the acquired image will be relatively low, which may lead to low accuracy of face recognition. Based on this, in order to further improve the accuracy of face recognition, in one embodiment, before the second recognition model is used to recognize each third image and obtain the target face feature vector corresponding to each third image, the face recognition method provided in the embodiment of the present application may also include the following steps:

[0156] obtaining an image resolution of each third image;

[0157] When the image resolution of the third image is less than the preset resolution threshold, the image resolution of the third image is adjusted until the image resolution of the adjusted third image is greater than or equal to the preset resolution threshold, thereby obtaining the adjusted third image.

[0158] The preset resolution threshold may be a resolution threshold preset based on actual experience or circumstances. For example, the preset resolution may be 50*50, which is not specifically limited herein.

[0159] Specifically, the face recognition device can obtain the image resolution of the third image and determine whether the image resolution of the third image is less than a preset resolution threshold. If the image resolution of the third image is less than the preset resolution threshold, the image resolution of the third image can be adjusted until the image resolution of the adjusted third image is greater than or equal to the preset resolution threshold. In this way, the face recognition device can obtain the adjusted third image.

[0160] More specifically, in the process of adjusting the image resolution of the third image, an Enhanced Super-Resolution Generative Adversarial Network (ESRGAN) may be used to adjust the image resolution of the third image.

[0161] In this embodiment, if the third image is a low-resolution image, the low-resolution image can be adjusted to a high-resolution image. In this way, the clarity and detail fidelity of the facial image are significantly improved, thereby providing higher quality input for subsequent facial recognition and further improving the accuracy of facial recognition.

[0162] In some embodiments, the above-mentioned preset correspondence relationship between the facial feature vector and the user identity information may include multiple user identity information and the facial feature vector corresponding to each user identity information;

[0163] It should be noted that the facial feature vector corresponding to each user identity information may include a first sub-feature vector and a second sub-feature vector, wherein the first sub-feature vector can be determined based on an occluded facial image corresponding to the user identity information, and the second sub-feature vector can be determined based on an unobstructed facial image corresponding to the user identity information.

[0164] In this embodiment, since in the correspondence between the preset facial feature vector and the user identity information, one user identity information corresponds to a feature vector determined based on an occluded facial image, and a feature vector determined based on an unobstructed facial image, in this way, during the face recognition process, the user identity information of the user can be accurately identified regardless of whether the user's facial area is obstructed.

[0165] In addition, since the face recognition model provided in the embodiment of the present application uses the first recognition model to recognize each first image, based on this, in order to accurately recognize the first image and obtain a more accurate recognition result of the first image, in some embodiments, the recognition result of the first image also includes the face confidence of the face area, the key point positions of multiple face key points in the face area, and the key point confidence of multiple face key points in the face area. Among them, the face confidence of the face area is used to characterize the probability that the second image contains the face area, and the key point confidence is used to characterize the reliability of the key point corresponding to the key point confidence.

[0166] Based on this, Figure 2 As shown, the above-mentioned S120 may specifically include the following steps:

[0167] S210, using the feature extraction layer in the first recognition model to extract image feature data of the first image, and using the attention mechanism to process the occluded face features and the unoccluded face features to obtain a first feature map.

[0168] S220, using the multi-scale feature fusion layer in the first recognition model to process the first feature map to obtain a plurality of second feature maps of different sizes, and fusing the plurality of second feature maps of different sizes to obtain a third feature map.

[0169] S230, using the first detection head in the first recognition model to process the third feature map, obtain the bounding box position of the bounding box including the face area and the face confidence of the face area, and based on the bounding box position of the bounding box including the face area, crop the first image to obtain the second image.

[0170] S240, using the second detection head in the first recognition model to process the third feature map to obtain key point positions of multiple facial key points in the face area and key point confidences of the multiple facial key points in the face area.

[0171] In some embodiments, the image feature data of the first image may include occluded facial features and unoccluded facial features.

[0172] Specifically, after acquiring N first images, the image processing device can, for each of the N first images, use the feature extraction layer in the first recognition model to extract image feature data of the first image. Since the image feature data of the first image includes occluded face features and unoccluded face features, the image processing device can use the attention mechanism to process the occluded face features and the unoccluded face features to obtain a first feature map, and then use the multi-scale feature fusion layer in the first recognition model to process the first feature map to obtain multiple second feature maps of different sizes, and fuse the multiple second feature maps of different sizes to obtain a third feature map, and then use the first detection head in the first recognition model to process the third feature map to obtain the bounding box position of the bounding box containing the face area and the face confidence of the face area. In this way, the first image can be cropped based on the bounding box position of the bounding box containing the face area to obtain the second image, and the third feature map can be processed by the second detection head in the first recognition model to obtain the key point positions of multiple face key points in the face area and the key point confidences of multiple face key points in the face area.

[0173] It should be noted here that in order to accurately identify the first image, the first recognition model in the image processing method provided in the embodiment of the present application is improved accordingly to better capture the subtle features of the face of a person wearing a mask. The specific improvements are as follows:

[0174] 1. Embed an attention mechanism in the feature extraction layer: This enables the first recognition model to pay more attention to the facial areas not blocked by the mask and reduce the weight of the areas blocked by the mask. This mechanism can make fuller use of the features not blocked by the mask and improve the accuracy of detection and key point prediction.

[0175] 2. Add a multi-scale feature fusion layer: Introduce multi-scale feature fusion technology, combine features of different scales for fusion processing, and enhance the network's robustness to faces of various sizes and postures.

[0176] 3. Optimize the detection head: The detection head is designed as a dual-task output structure, which outputs both the bounding box position of the face area and the position of the key points of the face. Specifically, we added a branch for key point detection in the detection head. By sharing the feature layer, we can reduce the amount of calculation and ensure the coordinated optimization of the two tasks. The key point detection branch can use high-resolution feature maps and combine the aforementioned attention mechanism to more accurately predict the key point position. Through high-resolution feature maps, the network can capture more detailed information and improve the accuracy of key point prediction.

[0177] In this embodiment, since the first recognition model involved in the embodiment of the present application includes a feature extraction layer, a multi-scale feature fusion layer, a detection head for detecting face areas, and a detection head for detecting face key points, in this way, in the process of processing the first image using the first recognition model, face detection and key point detection can be combined to accurately detect the face area and predict the position of the face key points, so that face recognition can be performed more accurately later.

[0178] Based on this, it should be noted that the image processing method provided in the embodiment of the present application needs to use a pre-trained first recognition model to process the first image. Therefore, before S120, the first recognition model needs to be trained first. Therefore, the specific implementation of the training method of the first recognition model provided in the embodiment of the present application is described below in conjunction with the accompanying drawings.

[0179] Figure 3 It is a flowchart of a training method for a first recognition model provided in an embodiment of the present application.

[0180] like Figure 3 As shown, the execution subject of the method may be an image processing device, based on which, the method may include the following steps:

[0181] S310: Obtain a first training sample set.

[0182] Before introducing this step, it should be noted that before training to obtain the first recognition model, the first preset recognition model needs to be iteratively processed multiple times to adjust its loss function value until the loss function value satisfies the first training adjustment condition to obtain the trained first recognition model. However, in each iterative training process, if only one first image sample is input, the sample size is too small to be conducive to the training adjustment of the first preset recognition model, that is, in the process of training the first preset recognition model, multiple first image samples are required to iteratively process the first preset recognition model to obtain the trained first recognition model.

[0183] Therefore, the first training sample set may include a plurality of first image samples and a sample label of each first image sample.

[0184] S320, for each first image sample, use the feature extraction layer of the first preset recognition model to extract the reference image feature data of the first image sample, and use the attention mechanism to process the reference occluded face features and the reference unoccluded face features to obtain a first reference feature map.

[0185] Among them, the above-mentioned first preset recognition model can be a related recognition model pre-set based on actual experience acquisition, and is not specifically limited here.

[0186] In some embodiments, the reference image feature data of the first image sample mentioned above may include reference occluded face features and reference unoccluded face features.

[0187] Specifically, after obtaining the first training sample set, since the first training sample set may include multiple first image samples, for each first image sample, the feature extraction layer of the first preset recognition model is used to extract the reference image feature data of the first image sample, and the reference image feature data may include reference occluded face features and reference unoccluded face features. In this way, the image processing device can use the attention mechanism to process the reference occluded face features and the reference unoccluded face features to obtain a first reference feature map.

[0188] S330, using the multi-scale feature fusion layer in the first preset recognition model to process the first reference feature map to obtain a plurality of second reference feature maps of different sizes, and fusing the plurality of second reference feature maps of different sizes to obtain a third reference feature map.

[0189] Specifically, after obtaining the first reference feature map, the first reference feature map can be processed using the multi-scale feature fusion layer in the first preset recognition model to obtain multiple second reference feature maps of different sizes, and then the multiple second reference feature maps can be fused to obtain the third reference feature map.

[0190] S340, using the first detection head in the first preset recognition model to process the third reference feature map, obtain the reference bounding box position of the reference bounding box including the face area and the reference face confidence of the face area, and based on the bounding box position of the reference bounding box including the face area, crop the first image sample to obtain a reference image.

[0191] Specifically, after obtaining the third reference feature map, the third feature map can be processed using the first detection head in the first preset recognition model to obtain the reference bounding box position of the reference bounding box including the face area and the reference face confidence of the above face area, and then based on the bounding box position of the reference bounding box including the face area, the first image sample is cropped to obtain a reference image, which includes the face area.

[0192] S350, using the second detection head in the first preset recognition model to process the third feature map, to obtain the reference key point positions of the multiple facial key points in the face area and the reference key point confidences of the multiple facial key points in the face area.

[0193] Among them, the above-mentioned reference key point confidence represents the reliability of the corresponding facial key point.

[0194] Specifically, after obtaining the third reference feature map, the image processing device can not only use the first detection head in the first preset recognition model to process the three feature maps, but the image processing device can also use the second detection head in the first preset recognition model to process the third feature map to obtain the reference key point position of each facial key point among the multiple facial key points in the face area and the reference key point confidence of each facial key point among the multiple facial key points in the face area.

[0195] S360, determining a loss function value of a first preset recognition model based on a reference recognition result of the first target training sample and a sample label of the first target training sample.

[0196] Among them, the reference recognition result of the above-mentioned first image sample may include a first reference image containing a face area and its corresponding reference face confidence, as well as reference key point positions of multiple face key points in the face area and their corresponding reference key point confidences.

[0197] In addition, the first target training sample may be any one of a plurality of first image samples.

[0198] Specifically, since the reference recognition result of the above-mentioned first image sample may include a first reference image including a face area and its corresponding reference face confidence, as well as reference key point positions of multiple face key points in the face area and their corresponding reference key point confidences, the image processing device can determine the loss function value of the first preset recognition model based on the reference recognition result of the first target training sample and the sample label of the first target training sample after obtaining the first reference image including a face area and its corresponding reference face confidence, as well as reference key point positions of multiple face key points in the face area and their corresponding reference key point confidences.

[0199] S370, based on the loss function value of the first preset recognition model, use multiple first image samples to train the first preset recognition model to obtain a trained first preset recognition model.

[0200] Specifically, in order to obtain a better first recognition model after training, the image processing device can adjust the model parameters of the first preset recognition model based on the loss function value of the first preset recognition model not satisfying the first training stop condition, and use multiple first image samples to train the first preset recognition model after the model parameters are adjusted, until the loss function value of the first preset recognition model satisfies the first training stop condition, and obtain the trained first recognition model. Among them, the first training stop condition can be based on actual experience or circumstances, and is not specifically limited here.

[0201] In this embodiment, a first training sample set can be obtained, which includes multiple first image samples and a sample label corresponding to each first image sample. Based on this, for each first image sample, the first image sample can be identified using a first preset recognition model to obtain a reference recognition result of the first image sample, and based on the reference recognition result of the first target image sample and the sample label of the first target image sample, the loss function value of the first preset recognition model can be determined. Furthermore, based on the loss function value of the first preset recognition model, the first preset recognition model can be trained using multiple first image samples to obtain a more accurate first recognition model.

[0202] Similarly, the face recognition method provided in the embodiment of the present application is to use the second recognition model to recognize each third image to obtain the target face feature vector corresponding to each third image. Based on this, in one embodiment, the above S140 may specifically include the following steps:

[0203] The image feature data of each third image is extracted using the second recognition model, and based on the image feature data of the third image, a target face feature vector corresponding to the third image is determined.

[0204] Specifically, the face recognition device can use the second recognition model to extract image feature data of the third image, and determine the target face feature vector corresponding to the third image based on the image feature data of the third image.

[0205] It should be noted here that the above-mentioned second recognition model includes a shared feature extraction layer and a task-specific layer. The second recognition model can extract image feature data of the third image through the shared feature extraction layer, and use the task-specific layer to process the image feature data of the third image to obtain the target face feature vector corresponding to the third image.

[0206] In this embodiment, the second recognition model can be used to extract image feature data of each third image, and based on the image feature data of the third image, the target facial feature vector corresponding to the third image can be accurately determined, so as to facilitate more accurate facial recognition later.

[0207] Based on this, it should be noted that the image processing method provided in the embodiment of the present application needs to use a pre-trained first recognition model to recognize the third image. Therefore, before S130, the second recognition model needs to be trained first. Therefore, the specific implementation of the image processing method provided in the embodiment of the present application is described below in conjunction with the accompanying drawings.

[0208] Figure 4 It is a flowchart of a training method for a second recognition model provided in an embodiment of the present application.

[0209] like Figure 4 As shown, the execution subject of the method may be an image processing device, based on which, the method may include the following steps:

[0210] S410: Obtain a second training sample set.

[0211] Before introducing this step, it should be noted that before obtaining the second recognition model, the second preset recognition model needs to be iteratively processed multiple times to adjust its loss function value until the loss function value satisfies the second training stop condition to obtain the trained second recognition model. However, in each iterative training, if only one second image sample is input, the sample size is too small to be conducive to the training adjustment of the second preset recognition model, that is, in the process of training the second preset recognition model, multiple second image samples are required to iteratively process the second preset recognition model to obtain the second recognition model.

[0212] Therefore, the second training sample set may include multiple second image samples and sample labels of each second image sample, and each second image sample includes a second image sample of a face region.

[0213] S420: For each second image sample, extract image feature data of the second image sample using a second preset recognition model, and determine a reference facial feature vector corresponding to the second image sample based on the image feature data of the second image sample.

[0214] The second preset recognition model may be a model pre-set based on actual experience or circumstances, which is not specifically limited here.

[0215] Specifically, after obtaining the second training sample set, the image processing device can include multiple second image samples, and each second image sample includes a second image sample containing a face area, so the image processing device can extract the image feature data of the second image sample using a second preset recognition model for each second image sample, and determine the reference face feature vector corresponding to the second image sample based on the image feature data of the second image sample.

[0216] S430, determining a loss function value of a second preset recognition model based on a reference facial feature vector corresponding to the second target training sample and a sample label of the second training target training sample.

[0217] The second target training sample may be any one of a plurality of second image samples.

[0218] Specifically, the image processing device can determine the loss function value of the second preset recognition model based on the reference facial feature vector corresponding to the second target training sample and the sample label of the second training target training sample.

[0219] S440, based on the loss function value of the second preset recognition model, use multiple second image samples to train the second preset recognition model to obtain a trained second recognition model.

[0220] Specifically, in order to obtain a better second recognition model, the image processing device can, after obtaining the loss function value of the second preset recognition model, adjust the model parameters of the second preset recognition model when it is determined that the loss function value of the second preset recognition model does not meet the second training stop condition, and use multiple second image samples to train the second preset recognition model after the model parameters are adjusted, until the loss function value of the second preset recognition model meets the second training stop condition, and obtain the trained second recognition model. The second training stop condition can be pre-set based on actual experience or circumstances, and is not specifically limited here.

[0221] In this embodiment, a second training sample set can be obtained, which includes multiple second image samples and a sample label corresponding to each second image sample. Based on this, for each second image sample, the second image sample is identified using a second preset recognition model to obtain a reference facial feature vector corresponding to the second image sample, and the reference facial feature vector corresponding to the second target image sample and the sample label of the second target image sample, and the loss function value of the second preset recognition model is determined, and then the loss function value of the second preset recognition model can be determined, and the second preset recognition model can be trained using multiple second image samples to obtain a more accurate second recognition model.

[0222] Based on this, after the first recognition model and the second recognition model are obtained through training, in order to fully and in detail describe the face recognition method provided by the embodiment of the present application, in a complete embodiment, as shown in FIG. Figure 5 As shown, the face recognition method may include the following steps:

[0223] S501, acquiring N first images.

[0224] S502: Use the first recognition model to recognize each first image to obtain a recognition result of each first image.

[0225] S503 , for each second image in the N second images, correct the face region based on key point positions of a plurality of face key points in the face region contained in the second image to obtain N corrected second images.

[0226] S504, selecting M third images from the N second images.

[0227] S505: Obtain a first resolution of each third image.

[0228] S506: When the image resolution of the third image is less than a preset resolution threshold, adjust the image resolution of the third image until the image resolution of the adjusted third image is greater than or equal to the preset resolution threshold, thereby obtaining an adjusted third image.

[0229] S507, using the second recognition model to recognize each third image, and obtaining a target face feature vector corresponding to each third image.

[0230] 509 , based on the preset correspondence between the facial feature vector and the user identity information, the target facial feature vector is matched to determine the target user identity information corresponding to the target facial feature vector.

[0231] In this embodiment, the face recognition device can obtain N first images, and use the first recognition model to recognize each of the first images to obtain the recognition result of each of the first images. Since the recognition result of each first image includes the second image containing the face area and the key point positions of multiple face key points in the face area, N second images can be obtained, and then the face area can be corrected for each of the N second images based on the key point positions of multiple face key points in the face area contained in the second image to obtain the corrected N second images, and then M third images are screened out from the N second images, and the image resolution of the third image is obtained. If the image resolution of the third image is less than the preset resolution threshold, the image resolution of the third image is adjusted until the image resolution of the adjusted third image is greater than or equal to the preset resolution threshold, and the adjusted third image is obtained. In this way, the second recognition model can be used to recognize each third image to obtain the target face feature vector corresponding to each of the third images, and the target face feature vector is matched based on the preset correspondence between the face feature vector and the user identity information to determine the target user identity information corresponding to the target face feature vector. In this way, accurate face recognition is achieved.

[0232] It should be noted that the face recognition method provided in the embodiments of the present application can be applied to a variety of practical scenarios, for example, public safety and monitoring; real-time monitoring and personnel identity recognition in large public places such as airports, stations, and shopping malls to improve public safety; identity authentication and abnormal behavior detection in sensitive areas such as schools and hospitals; smart home and access control systems; used for home security and smart door locks to automatically identify family members and visitors, and improve home safety and convenience; realize contactless identity authentication in enterprise access control systems to improve the level of enterprise security management; smart transportation; used for passenger identity authentication on public transportation such as buses and subways to improve the safety and convenience of riding; used for passenger identity authentication and security monitoring in driverless cars.

[0233] Based on the same inventive concept, the embodiment of the present application also provides a face recognition device. Figure 6 The face recognition device provided in the embodiment of the present application is described in detail.

[0234] Figure 6 It is a structural schematic diagram of a face recognition device provided in an embodiment of the present application.

[0235] like Figure 6 As shown, the face recognition device 600 may include:

[0236] An acquisition module 610 is used to acquire N first images, where N is a positive integer;

[0237] The recognition module 620 is used to recognize each first image using the first recognition model to obtain a recognition result of each first image; the recognition result of the first image includes a second image containing a face area;

[0238] A screening module 630, configured to screen out M third images from the N second images, where M is a positive integer less than or equal to N;

[0239] The recognition module 620 is further used to recognize each third image using the second recognition model to obtain a target face feature vector corresponding to each third image;

[0240] The matching module 640 is used to match each target facial feature vector based on the preset correspondence between the facial feature vector and the user identity information, and identify the target user identity information corresponding to each target facial feature vector.

[0241] In one embodiment, the above-mentioned screening module is specifically used to: screen out M third images that meet preset conditions from N second images based on at least one of face confidence, face posture information and image quality information; face confidence is used to characterize the probability that the second image contains a face area.

[0242] In one embodiment, the image quality information includes at least one of image clarity, image brightness, and image contrast;

[0243] In the case where the image quality information includes image clarity, the preset condition is that the image clarity is greater than a preset clarity threshold; or,

[0244] In the case where the image quality information includes image brightness, the preset condition is that the image brightness is within a preset brightness range; or,

[0245] In the case where the image quality information includes image contrast, the preset condition is that the image contrast is within a preset contrast range.

[0246] In one embodiment, the preset condition includes that the face confidence is greater than a preset confidence threshold; or,

[0247] The face posture information includes at least one of a yaw angle, a pitch angle, and a roll angle;

[0248] When the face posture information includes a yaw angle, the preset condition is that the yaw angle is within a first angle range; or,

[0249] When the face posture information includes a pitch angle, the preset condition is that the pitch angle is within a second angle range; or,

[0250] When the facial posture information includes a roll angle, the preset condition is that the roll angle is within a third angle range.

[0251] In one embodiment, the preset condition is that the face confidence is greater than a preset confidence threshold.

[0252] In one embodiment, the facial posture information includes at least one of a yaw angle, a pitch angle, and a roll angle;

[0253] When the face posture information includes a yaw angle, the preset condition is that the yaw angle is within a first angle range; or,

[0254] When the face posture information includes a pitch angle, the preset condition is that the pitch angle is within a second angle range; or,

[0255] When the facial posture information includes a roll angle, the preset condition is that the roll angle is within a third angle range.

[0256] In one embodiment, the recognition result of the first image further includes key point positions of multiple key points of the face in the face region. Based on this, the face recognition device provided in the embodiment of the present application may further include a correction module.

[0257] The correction module is used to correct the face area of ​​each second image in the N second images based on the key point positions of multiple face key points in the face area contained in the second image to obtain the corrected N second images.

[0258] In one embodiment, the face recognition device provided by the embodiment of the present application may also include an adjustment module.

[0259] The acquisition module is further used to acquire the image resolution of each of the third images;

[0260] The adjustment module is used to adjust the image resolution of the third image when the image resolution of the third image is less than a preset resolution threshold, until the image resolution of the adjusted third image is greater than or equal to the preset resolution threshold, thereby obtaining the adjusted third image.

[0261] In one embodiment, the preset correspondence relationship between the facial feature vector and the user identity information includes a plurality of user identity information and a facial feature vector corresponding to each user identity information;

[0262] The facial feature vector corresponding to each user identity information includes a first sub-feature vector and a second sub-feature vector. The first sub-feature vector is determined based on the obstructed facial image corresponding to the user identity information, and the second sub-feature vector is determined based on the unobstructed facial image corresponding to the user identity information.

[0263] In the embodiment of the present application, N first images can be obtained, and each first image can be preliminarily recognized using the first recognition model to obtain the recognition result of each first image, and the recognition result of the first image includes a second image containing a face area, and then M third images with higher quality can be screened out from the N second images, and each third image can be further recognized using the second recognition model to obtain the target face feature vector corresponding to each third image, and then each target face feature vector can be matched based on the correspondence between the preset face feature vector and the user identity information to accurately identify the target user identity information corresponding to each target face feature vector. In this way, by using the first recognition model to perform initial recognition on the image, a face image containing a face area is obtained, and then the second model is used to perform face recognition again on the image containing the face area, thereby improving the face recognition accuracy when the face area is small, and in this process, by screening some images from the acquired multiple images for final face recognition, not only the face recognition accuracy is further improved, but also the resource consumption is reduced.

[0264] Each module in the face recognition device provided in the embodiment of the present application can realize Figures 1 to 5 The method steps of any of the embodiments shown in the figure can achieve the corresponding technical effects, which will not be repeated here for the sake of brevity.

[0265] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0266] The electronic device may include a processor 701 and a memory 702 storing computer program instructions.

[0267] Specifically, the processor 701 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0268] The memory 702 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 702 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 702 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 702 may be inside or outside a comprehensive electronic device. In a particular embodiment, the memory 702 is a non-volatile solid-state memory.

[0269] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0270] The processor 701 implements any one of the face recognition methods in the above embodiments by reading and executing computer program instructions stored in the memory 702 .

[0271] In one example, the electronic device may further include a communication interface 703 and a bus 710. Figure 7 As shown, the processor 701, the memory 702, and the communication interface 703 are connected via a bus 710 and communicate with each other.

[0272] The communication interface 703 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0273] Bus 710 includes hardware, software or both, and the parts of electronic equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 710 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0274] In addition, in combination with the face recognition method in the above embodiment, the embodiment of the present application can provide a computer storage medium to implement. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, the face recognition method provided in the embodiment of the present application is implemented.

[0275] An embodiment of the present application also provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the face recognition method provided in the embodiment of the present application.

[0276] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0277] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0278] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0279] Aspects of the present disclosure are described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable face recognition device to produce a machine so that these instructions executed by the processor of the computer or other programmable face recognition device enable the implementation of the functions / actions specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0280] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the described systems, modules and units can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A method for face recognition, characterized in that: The method comprises: Obtain N first images, where N is a positive integer; Using a first recognition model to recognize each of the first images, to obtain a recognition result of each of the first images; the recognition result of the first images includes a second image containing a face area; Selecting M third images from the N second images, where M is a positive integer less than or equal to N; Using the second recognition model to recognize each of the third images, to obtain a target face feature vector corresponding to each of the third images; Based on the preset correspondence between the facial feature vector and the user identity information, each of the target facial feature vectors is matched to identify the target user identity information corresponding to each of the target facial feature vectors.

2. The method according to claim 1, characterized in that The step of selecting M third images from N second images comprises: Based on at least one of face confidence, face posture information and image quality information, M third images that meet preset conditions are screened out from N second images; the face confidence is used to characterize the probability that the second image contains a face area.

3. The method according to claim 2, characterized in that The image quality information includes at least one of image clarity, image brightness, and image contrast; In the case where the image quality information includes image clarity, the preset condition is that the image clarity is greater than a preset clarity threshold; or, In the case where the image quality information includes image brightness, the preset condition is that the image brightness is within a preset brightness range; or, In a case where the image quality information includes image contrast, the preset condition is that the image contrast is within a preset contrast range.

4. The method according to claim 3, characterized in that The preset condition includes that the face confidence is greater than a preset confidence threshold; or, The facial posture information includes at least one of a yaw angle, a pitch angle, and a roll angle; In the case where the face posture information includes the yaw angle, the preset condition is that the yaw angle is within a first angle range; or, In the case where the face posture information includes the pitch angle, the preset condition is that the pitch angle is within a second angle range; or, When the facial posture information includes the roll angle, the preset condition is that the roll angle is within a third angle range.

5. The method according to claim 2, characterized in that: The preset condition is that the face confidence is greater than a preset confidence threshold.

6. The method according to claim 2 or 5, characterized in that: The facial posture information includes at least one of a yaw angle, a pitch angle, and a roll angle; In the case where the face posture information includes the yaw angle, the preset condition is that the yaw angle is within a first angle range; or, In the case where the face posture information includes the pitch angle, the preset condition is that the pitch angle is within a second angle range; or, When the facial posture information includes the roll angle, the preset condition is that the roll angle is within a third angle range.

7. The method according to any one of claims 2 to 6, characterized in that: The recognition result of the first image also includes key point positions of multiple facial key points in the face area; Before selecting M third images meeting a preset condition from N second images based on at least one of face confidence, face posture information, and image quality information, the method further includes: For each of the N second images, the face region is corrected based on key point positions of a plurality of face key points in the face region contained in the second image to obtain N corrected second images.

8. The method according to claim 1, characterized in that Before using the second recognition model to recognize each of the third images to obtain a target face feature vector corresponding to each of the third images, the method further includes: Acquire the image resolution of each of the third images; When the image resolution of the third image is less than a preset resolution threshold, the image resolution of the third image is adjusted until the image resolution of the adjusted third image is greater than or equal to the preset resolution threshold, thereby obtaining an adjusted third image.

9. The method according to claim 1, characterized in that: The correspondence between the preset facial feature vector and the user identity information includes a plurality of user identity information and a facial feature vector corresponding to each of the user identity information; The facial feature vector corresponding to each user identity information includes a first sub-feature vector and a second sub-feature vector, the first sub-feature vector is determined based on the obstructed facial image corresponding to the user identity information, and the second sub-feature vector is determined based on the unobstructed facial image corresponding to the user identity information.

10. A face recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring N first images, where N is a positive integer; A recognition module, configured to recognize each of the first images using a first recognition model to obtain a recognition result of each of the first images; the recognition result of the first image includes a second image containing a face area; A screening module, used for screening out M third images from N second images, where M is a positive integer less than or equal to N; The recognition module is further used to recognize each of the third images using the second recognition model to obtain a target face feature vector corresponding to each of the third images; The matching module is used to match each of the target facial feature vectors based on the corresponding relationship between the preset facial feature vectors and the user identity information, and identify the target user identity information corresponding to each of the target facial feature vectors.

11. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; The processor reads and executes the computer program instructions to implement the face recognition method as described in any one of claims 1-9.

12. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, which, when executed by a processor, implement the face recognition method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method for face recognition described in any one of claims 1 to 9 is implemented.

Images