Face Image Screening Method and Device

Abstract

The present disclosure provides a method for screening face images, a device for screening face images, an electronic device, and a computer-readable storage medium. The method for screening face images includes: obtaining one or more original face images; extracting reconstruction feature information of the original face images; based on the reconstruction feature information, obtaining restored face images through feature reconstruction; determining the similarity between each original face image and the restored face image corresponding to the original face image; and determining qualified images in the original face images based on the similarity corresponding to each original face image. By extracting features based on face images, reconstructing based on the extracted features to obtain restored face images, and then comparing the similarity with the original face images to judge the quality of the original face images, the core problem of feature extraction related to face image quality is end-to-end processed, and a digital index is provided for face quality judgment.

Description

Technical Field

[0001] The present disclosure generally relates to the field of image processing, and particularly to a method for screening face images, a device for screening face images, an electronic device, and a computer-readable storage medium. Background Art

[0002] Currently, with the development of society and the progress of technology, in many scenarios, it is necessary to detect and identify a person object through a face image.

[0003] A face image is high-dimensional data, and face detection and face recognition are achieved through face features. Face features are extracted by a neural network to realize the conversion from high-dimensional data to low-dimensional features. In face detection and recognition tasks, the quality of a face image has a great impact on the extracted features. Features extracted from high-quality face images have high robustness and can improve the performance of tasks such as detection and recognition; low-quality face images will affect feature extraction and the accuracy of conclusions. However, in actual application scenarios, there is a lack of effective screening means to judge the quality of face images. Summary of the Invention

[0004] To solve the above problems existing in the prior art, a first aspect of the present disclosure provides a method for screening face images, where the method includes: obtaining one or more original face images; extracting reconstruction feature information of the original face images; based on the reconstruction feature information, obtaining a restored face image through feature reconstruction; determining the similarity between each original face image and the restored face image corresponding to the original face image; and determining qualified images in the original face images based on the similarity corresponding to each original face image.

[0005] In one example, extracting the reconstruction feature information of the original face images includes: performing feature extraction on the original face images to obtain face key point information; and based on the face key point information, obtaining the reconstruction feature information, where the reconstruction feature information includes: face pose information and UV map information.

[0006] In one example, based on the reconstruction feature information, obtaining a restored face image through feature reconstruction includes: inputting the face pose information and UV map information of the original face image into a neural network to obtain the restored face image corresponding to the original face image, where the restored face image is a frontal face image.

[0007] In one example, determining the similarity between each original face image and the restored face image corresponding to the original face image includes: respectively extracting a first feature vector of the original face image and a second feature vector of the restored face image corresponding to the original face image; and determining the similarity according to the distance between the first feature vector and the second feature vector.

[0008] In one example, determining qualified images among the original face images based on the similarity corresponding to each original face image includes: if the similarity corresponding to an original face image is greater than the similarity threshold, the original face image is a qualified image.

[0009] In one example, determining qualified images among the original face images based on the similarity corresponding to each original face image includes: sorting the original face images in descending order of similarity; selecting the top preset number of original face images as qualified images according to the preset number of qualified images.

[0010] In one example, obtaining one or more original face images includes: obtaining a video, where the video includes consecutive video frames; performing object detection on the video frames and obtaining one or more original face images through object tracking.

[0011] A second aspect of the present disclosure provides a face image screening device, which includes: an acquisition module for acquiring one or more original face images; a feature extraction module for extracting reconstruction feature information of the original face images; a reconstruction module for obtaining restored face images through feature reconstruction based on the reconstruction feature information; a comparison module for determining the similarity between each original face image and the restored face image corresponding to the original face image; a screening module for determining qualified images among the original face images based on the similarity corresponding to each original face image.

[0012] A third aspect of the present disclosure provides an electronic device, which includes: a memory for storing instructions; and a processor for calling the instructions stored in the memory to execute the face image screening method as in the first aspect.

[0013] A fourth aspect of the present disclosure provides a computer-readable storage medium, in which instructions are stored, and when the instructions are executed by a processor, the face image screening method as in the first aspect is executed.

[0014] The face image screening method, face image screening device, electronic device, and computer-readable storage medium provided by the present disclosure perform end-to-end processing on the core issue of feature extraction related to face image quality by extracting features based on face images, reconstructing based on the extracted features to obtain restored face images, and then comparing the similarity with the original face images to judge the quality of the original face images, and provide a digital index for face quality judgment. Description of the Drawings

[0015] By reading the following detailed description with reference to the accompanying drawings, the above and other purposes, features, and advantages of the embodiments of the present disclosure will become easy to understand. In the drawings, several embodiments of the present disclosure are shown in an exemplary rather than restrictive manner, where:

[0016] Figure 1 shows a schematic flow diagram of a face image screening method according to an embodiment of the present disclosure;

[0017] Figure 2 shows a schematic flow diagram of a face image screening method according to another embodiment of the present disclosure;

[0018] Figure 3 shows a schematic flow diagram of a face image screening method according to another embodiment of the present disclosure;

[0019] Figure 4 shows a schematic flow diagram of a face image screening method according to another embodiment of the present disclosure;

[0020] Figure 5 shows a schematic diagram of a face image screening device according to an embodiment of the present disclosure.

[0021] Figure 6 is a schematic diagram of an electronic device provided by an embodiment of the present disclosure.

[0022] In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Embodiments

[0023] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and do not limit the scope of the present disclosure in any way.

[0024] It should be noted that although terms such as "first" and "second" are used in this document to describe different modules, steps, and data of the embodiments of the present disclosure, these terms are only used to distinguish between different modules, steps, and data, and do not represent a specific order or importance. In fact, the terms "first" and "second" can be used interchangeably.

[0025] In some related technologies, the quality of a face image is directly judged by the size and angle of the face in the face image, but this method is not accurate and objective. The size and angle of the face are not the main factors affecting the extraction of features by the neural network, and they cannot truly and comprehensively reflect the quality of the face image.

[0026] In some other related technologies, the method for evaluating the quality of a face image is based on face key points (landmarks). After performing a warp operation on the face image to obtain face key points again, the differences between the two face key points are compared to judge the quality of the face image. The disadvantage of this method is that for faces with a large deflection angle, the judgment result is not accurate, and at the same time, the warp operation is also very cumbersome.

[0027] To solve the above problems, a method for accurately, directly, and objectively evaluating the quality of face images is provided. Figure 1 FIG. 10 shows a face image screening method 10 provided by an embodiment of the present disclosure, including: step S11-step S15. The above steps will be described in detail below:

[0028] Step S11: Obtain one or more original face images.

[0029] According to actual needs, one original face image can be obtained to determine whether the quality of the image is qualified; or multiple original face images can be obtained to screen out the qualified images or the images with the best quality among them.

[0030] In one embodiment, as Figure 2 shown, step S11 may include: step S111, obtain a video, where the video includes consecutive video frames; step S112, perform object detection on the video frames, and through object tracking, obtain one or more original face images. In this embodiment, a video can be obtained. Among the consecutive video frames, there are people. Through object detection, images of the human body, face, or human head can be detected. For the same person, the face image of this person in each frame of the consecutive video frames can be obtained through object tracking (track). In actual application scenarios, multiple face images obtained are often used for subsequent tasks, such as judging the attribute information of a person through a neural network model or re-identifying based on a gallery image, etc. When performing subsequent tasks, the quality of the face image directly affects the result of the task. Therefore, through the face image screening method 10 provided by the present disclosure, qualified images or the best one or more images can be selected from multiple face images of the same person, and then subsequent tasks are performed, thereby ensuring the accuracy of the subsequent tasks.

[0031] In some other embodiments, the original face image can also be a picture of the same person in an existing portrait file. Through screening by the face image screening method 10, the images in the file can be streamlined, the quality of the images in the file can be ensured, and the accuracy can be ensured when used for subsequent recognition applications or as a gallery image.

[0032] Step S12: Extract the reconstruction feature information of the original face image.

[0033] There are various ways to extract features. In some embodiments, the reconstruction feature information can be obtained by extracting features from the original face image through a neural network for subsequent reconstruction to obtain a restored face image.

[0034] In one embodiment, as Figure 3As shown, step S12 may include: step S121, performing feature extraction on the original face image to obtain face key point information; step S122, obtaining reconstructed feature information based on the face key point information, where the reconstructed feature information includes: face pose information and UV map information. In this embodiment, the original face image may be subjected to feature extraction through a neural network to obtain face key point information. Among them, in one example, the face key points may be dense key points. For example, 128 key points are obtained for the axis, eyes, and other parts of the face, which can ensure comprehensive features and the accuracy of subsequent processing. In another example, the face key points may also be sparse key points, which can locate the facial features and pose, and can meet the requirements of the present disclosure, thereby effectively reducing the computational amount and operating cost. After obtaining the face key point information, face pose information and UV map information are obtained based on the face key point information. Among them, the face pose information reflects the pose of the face, mainly including three angle information, that is, the rotation angles of the three axes of the face in three-dimensional space; and the UV map information is the unfolded mapping image of the three-dimensional face image in two-dimensional space.

[0035] Step S13, based on the reconstructed feature information, obtain a restored face image through feature reconstruction.

[0036] Through the reconstructed feature information, feature reconstruction can be performed through another neural network to obtain a restored face image, which is reconstructed based on the features of the original face image and corresponds to the original face image.

[0037] In one embodiment, step S13 may include inputting the face pose information and UV map information of the original face image into a neural network to obtain a restored face image corresponding to the original face image, where the restored face image is a frontal face image. In this embodiment, based on the face pose information and UV map information, a frontal face image can be obtained through a neural network, thereby ensuring the quality of the reconstructed image and the accuracy of subsequent feature extraction, and then ensuring the reliability of the screening result.

[0038] Step S14, determine the similarity between each original face image and the restored face image corresponding to the original face image.

[0039] For high-quality original portrait images, features with high robustness should be able to be extracted. Therefore, they should be similar to the reconstructed frontal face image. Therefore, determining the similarity between the original face image and the restored face image reconstructed therefrom can reasonably judge and screen the quality of the original face image.

[0040] In one embodiment, such as Figure 4As shown, step S14 may further include: step S141, respectively extracting a first feature vector of the original face image and a second feature vector of the restored face image corresponding to the original face image; step S142, determining the similarity according to the distance between the first feature vector and the second feature vector. In this embodiment, a face recognition model or other feature extraction model may be used to extract features from the original face image and the corresponding restored face image. According to the model structure, multi-dimensional feature vectors representing face features can be obtained respectively, and the distance between the two feature vectors is determined. The closer the distance, the higher the similarity. Among them, the distance can be calculated using the Euclidean distance. Determining the similarity through the distance of the feature vectors can objectively and accurately evaluate whether the original face image and the restored face image are similar. Moreover, the method provided in the present disclosure can obtain more accurate results when determining whether the original face image is performing feature extraction in many scenarios. Therefore, the result obtained by judging through the feature vectors obtained by feature extraction is more direct and accurate.

[0041] Step S15, based on the similarity corresponding to each original face image, determine the qualified images in the original face images.

[0042] Since the restored face image is reconstructed from the features extracted from the original face image, therefore, the similarity between the two means that the features extracted from the original face image have strong robustness, which also means that the original face image belongs to a high-quality image in the scenario of image feature extraction.

[0043] In one embodiment, step S15 may include: if the similarity corresponding to the original image is greater than the similarity threshold, then the original face image is a qualified image. In this embodiment, by setting the similarity threshold, the original face images with similarity greater than the similarity threshold can be determined as qualified images, which can be used for subsequent tasks to ensure the reliability of subsequent tasks. This embodiment is applicable to quality screening, eliminating low-quality original face images and retaining high-quality qualified images.

[0044] In another embodiment, step S15 may include: sorting the original face images in descending order of similarity; according to the preset number of qualified images, selecting the preset number of original face images ranked at the top as qualified images. In this embodiment, the original face images are sorted according to similarity, and according to the preset number, one or more with the highest similarity are selected as qualified images. The preset number can be a fixed value, such as 3 or 5, etc., or it can be a ratio, that is, a value is obtained as the number of qualified images according to the ratio of the total number of original face images involved in this screening, such as 10% or 5%, etc. This embodiment is applicable to selecting the best from multiple original face images to obtain one or more original images with the highest quality that can best represent the person for subsequent tasks.

[0045] In some other embodiments, the above two methods for determining qualified images can be combined. For example, first perform sorting, preliminarily select a number of optimal original face images according to a preset quantity, and then perform screening according to a similarity threshold. If there are still original face images that do not meet the similarity threshold, they will also be excluded.

[0046] In the foregoing embodiments, after screening, the original face images that do not meet the qualified image standard can be fed back, and then can be processed by means such as deletion or separate filing.

[0047] Based on the same inventive concept, the present disclosure also provides a face image screening device 100, as Figure 5 shown. The face image screening device 100 includes: an acquisition module 110, configured to acquire one or more original face images; a feature extraction module 120, configured to extract reconstruction feature information of the original face images; a reconstruction module 130, configured to obtain a restored face image through feature reconstruction based on the reconstruction feature information; a comparison module 140, configured to determine the similarity between each original face image and the restored face image corresponding to the original face image; and a screening module 150, configured to determine qualified images in the original face images based on the similarity corresponding to each original face image.

[0048] In one embodiment, the feature extraction module 120 is further configured to: perform feature extraction on the original face images to obtain face key point information; and obtain reconstruction feature information based on the face key point information, where the reconstruction feature information includes: face pose information and UV mapping information.

[0049] In one embodiment, the reconstruction module 130 is further configured to: input the face pose information and UV mapping information of the original face images into a neural network to obtain a restored face image corresponding to the original face images, where the restored face image is a frontal face image.

[0050] In one embodiment, the comparison module 140 is further configured to: respectively extract a first feature vector of the original face image and a second feature vector of the restored face image corresponding to the original face image; and determine the similarity according to the distance between the first feature vector and the second feature vector.

[0051] In one embodiment, the screening module 150 is further configured to: if the similarity corresponding to the original face image is greater than the similarity threshold, then the original face image is a qualified image.

[0052] In one embodiment, the screening module 150 is further configured to: sort the original face images in descending order of similarity; and select the preset number of original face images ranked at the top as qualified images according to the preset quantity of qualified images.

[0053] In one embodiment, the obtaining module 110 is further configured to: obtain a video, where the video includes consecutive video frames; perform object detection on the video frames, and obtain one or more original face images through object tracking.

[0054] Regarding the face image screening device 100 in the above embodiment, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0055] As Figure 6 shown, an embodiment of the present disclosure provides an electronic device 200. Among them, the electronic device 200 includes a memory 201, a processor 202, and an input / output (I / O) interface 203. Among them, the memory 201 is used to store instructions. The processor 202 is used to call the instructions stored in the memory 201 to execute the face image screening method of the embodiments of the present disclosure. Among them, the processor 202 is respectively connected to the memory 201 and the I / O interface 203, and can be connected through a bus system and / or other forms of connection mechanisms (not shown) for example. The memory 201 can be used to store programs and data, including the program of the face image screening method involved in the embodiments of the present disclosure. The processor 202 executes various functional applications and data processing of the electronic device 200 by running the program stored in the memory 201.

[0056] In the embodiments of the present disclosure, the processor 202 can be implemented in at least one of the hardware forms of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 202 can be a central processing unit (CPU) or a combination of one or more of other forms of processing units with data processing capabilities and / or instruction execution capabilities.

[0057] The memory 201 in the embodiments of the present disclosure may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.

[0058] In the embodiments of the present disclosure, the I / O interface 203 can be used to receive input instructions (such as digital or character information, and key signal inputs related to the user settings and function controls of the electronic device 200, etc.), and can also output various information to the outside (such as images or sounds, etc.). In the embodiments of the present disclosure, the I / O interface 203 may include one or more of a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a mouse, a joystick, a trackball, a microphone, a speaker, and a touch panel, etc.

[0059] It can be understood that although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring to perform these operations in the specific order or serial order shown, or requiring to perform all the operations shown to obtain the desired result. In a specific environment, multi-tasking and parallel processing may be beneficial.

[0060] The methods and apparatuses related to the embodiments of the present disclosure can be completed by using standard programming techniques, and various method steps are implemented by using rule-based logic or other logics. It should also be noted that the terms "apparatus" and "module" used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving inputs.

[0061] Any of the steps, operations, or programs described herein can be executed or implemented using one or more hardware or software modules alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product including a computer-readable medium containing computer program code, which can be executed by a computer processor to execute any or all of the described steps, operations, or programs.

[0062] For purposes of illustration and description, the foregoing description of the implementation of the present disclosure has been given. The foregoing description is not exhaustive and is not intended to limit the present disclosure to the exact forms disclosed, and various variations and modifications are possible in light of the above teachings, or various variations and modifications may be obtained from the practice of the present disclosure. These embodiments are chosen and described in order to illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can utilize the present disclosure in various embodiments and various modifications suitable for the specific purposes contemplated.

Claims

1. A method for screening face images, characterized in that, The method includes: Obtaining one or more original face images; Performing feature extraction on the original face images to obtain face key point information; Obtaining face pose information and UV mapping information based on the face key point information; Inputting the face pose information and the UV mapping information of the original face images into a neural network to obtain a restored face image corresponding to the original face images; wherein, the restored face image is a frontal face image; determining the similarity between each original face image and the restored face image corresponding to the original face image; Determining the qualified images among the original face images based on the similarity corresponding to each original face image.

2. The method according to claim 1, characterized in that The determining the similarity between each original face image and the restored face image corresponding to the original face image includes: Extracting a first feature vector of the original face image and a second feature vector of the restored face image corresponding to the original face image respectively; Determining the similarity according to the distance between the first feature vector and the second feature vector.

3. The method according to claim 1, characterized in that, The determining the qualified images among the original face images based on the similarity corresponding to each original face image includes: If the similarity corresponding to the original face image is greater than a similarity threshold, then the original face image is a qualified image.

4. The method according to claim 1, wherein The determining the qualified images among the original face images based on the similarity corresponding to each original face image includes: Sorting the original face images in descending order according to the similarity; Selecting the top preset number of the original face images as the qualified images according to the preset number of the qualified images.

5. The method according to claim 1, wherein The obtaining one or more original face images includes: Obtaining a video, where the video includes consecutive video frames; Performing object detection on the video frames and obtaining the one or more original face images through object tracking.

6. A face image screening device, characterized in that, The device includes: An obtaining module, configured to obtain one or more original face images; A feature extraction module, configured to perform feature extraction on the original face images to obtain face key point information; obtaining face pose information and UV mapping information based on the face key point information; A reconstruction module, configured to input the face pose information and the UV mapping information of the original face images into a neural network to obtain a restored face image corresponding to the original face images; wherein, the restored face image is a frontal face image; A comparison module, configured to determine the similarity between each original face image and the restored face image corresponding to the original face image; A screening module, configured to determine the qualified images among the original face images based on the similarity corresponding to each original face image.

7. An electronic device, characterized in that, The electronic device includes: A memory, configured to store instructions; and A processor, configured to call the instructions stored in the memory to execute the face image screening method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, Wherein instructions are stored, and when the instructions are executed by the processor, the face image screening method according to any one of claims 1-5 is executed.

Images