A face recognition method, device, equipment and storage medium

Abstract

This application provides a face recognition method, apparatus, device, and storage medium, relating to the field of artificial intelligence technology. The method includes: extracting target face features from a face image to be recognized in a target state dimension, and auxiliary face features corresponding to the face image to be recognized in at least one auxiliary state dimension; then, based on the target face features and the obtained auxiliary face features, determining whether the target object belongs to a specified identity type. Compared to face features in a single state dimension, face features in multiple state dimensions can more comprehensively represent different states of a face. Therefore, determining whether a target object belongs to a specified identity type based on face features in multiple state dimensions can effectively improve the accuracy of face recognition, thereby improving the efficiency and success rate of searching for missing persons.

Description

[0001] This application claims priority to Chinese Patent Application No. 202110497790.9, filed on May 8, 2021, entitled "A Face Recognition Method, Apparatus, Device and Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a face recognition method, device, equipment and storage medium. Background Technology

[0003] Finding missing persons has been a long-standing problem in this century. Helping missing persons return to their families is of great significance for both family stability and social harmony.

[0004] Currently, related technologies passively search for missing persons by posting missing persons information on websites. However, this approach has limited reach, with relatively few users seeing the posted missing persons information, resulting in low efficiency and success rates in searching for missing persons. Summary of the Invention

[0005] This application provides a face recognition method, apparatus, device, and storage medium to improve the efficiency and success rate of searching for missing persons.

[0006] On one hand, embodiments of this application provide a face recognition method, including: Acquire the image of the face to be identified from the target object; Extract the target face features from the face image to be identified, and determine the target state dimension corresponding to the face image to be identified based on the target face features; Based on the target face features and the target state dimension, predict the auxiliary face features corresponding to at least one auxiliary state dimension of the face image to be identified; Based on the target facial features and the obtained auxiliary facial features, determine whether the target object belongs to the specified identity type.

[0007] On one hand, embodiments of this application provide a face recognition device, including: The acquisition module is used to acquire the face image of the target object to be identified; The feature extraction module is used to extract the target face features of the face image to be identified, and to determine the target state dimension corresponding to the face image to be identified based on the target face features. The feature expansion module is used to predict the auxiliary face features corresponding to at least one auxiliary state dimension of the face image to be identified based on the target face features and the target state dimension. The discrimination module is used to determine whether the target object belongs to a specified identity type based on the target face features and the obtained auxiliary face features.

[0008] Optionally, the discrimination module is specifically used for: Obtain the facial depth image of the target object, and extract the facial depth features from the facial depth image; Based on the target facial features and the facial depth features, the comprehensive facial features of the target object are obtained; Based on the comprehensive facial features and the obtained auxiliary facial features, it is determined whether the target object belongs to the specified identity type.

[0009] Optionally, the discrimination module is specifically used for: If at least one of the comprehensive facial features and the obtained auxiliary facial features matches the facial features of the first reference facial image in the first facial image library, then the target object is determined to be of a specified identity type, wherein the first reference facial image is the facial image corresponding to the specified identity type. If none of the integrated facial features and the obtained auxiliary facial features match the facial features of the first reference facial image in the first facial image library, then the target object is determined not to be of the specified identity type.

[0010] Optionally, the discrimination module is further configured to: Before determining whether the target object is of a specified identity type based on the comprehensive facial features and the obtained auxiliary facial features, the infrared image of the target object's face is acquired; Extract the facial infrared features from the facial infrared image and determine that the facial infrared features meet the conditions for liveness detection.

[0011] Optionally, the feature extraction module is further configured to: Before extracting the target face features from the face image to be identified, it is determined that the image quality of the face image to be identified meets a first preset condition; Before extracting the facial depth features from the facial depth image, it is determined that the integrity of the facial depth image meets a second preset condition. Before extracting the facial infrared features from the facial infrared image, it is determined that the brightness of the facial infrared image meets a third preset condition.

[0012] Optionally, the discrimination module is specifically used for: If at least one of the target facial features and the obtained auxiliary facial features matches the facial features of the first reference facial image in the first facial image library, then the target object is determined to be of a specified identity type, wherein the first reference facial image is the facial image corresponding to the specified identity type. If none of the target facial features and the obtained auxiliary facial features match the facial features of the first reference facial image in the first facial image library, then the target object is determined not to be of the specified identity type.

[0013] Optionally, the discrimination module is further configured to: Before determining the target state dimension corresponding to the face image to be identified based on the target face features, it is determined that the target face features do not match the face features of the second reference face image in the second face image library, wherein the second reference face image is a face image corresponding to a non-specified identity type.

[0014] Optionally, the specified identity type is missing persons; Optionally, the discrimination module is further configured to: After determining that the target object is of a specified identity type, the target acquisition location and target acquisition time of the face image to be identified are obtained.

[0015] On one hand, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described face recognition method.

[0016] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described face recognition method.

[0017] In this embodiment, target facial features in the target state dimension and auxiliary facial features in at least one auxiliary state dimension of the face image to be identified are extracted. Then, based on the target facial features and the obtained auxiliary facial features, it is determined whether the target object belongs to a specified identity type. Compared with facial features in a single state dimension, facial features in multiple state dimensions can more comprehensively represent different states of the face. Therefore, determining whether the target object belongs to a specified identity type based on facial features in multiple state dimensions can effectively improve the accuracy of face recognition, thereby improving the efficiency and success rate of searching for missing persons. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A schematic diagram of a system architecture provided for an embodiment of this application; Figure 2 A flowchart illustrating a face recognition method provided in an embodiment of this application; Figure 3 A schematic diagram of a human face image provided in an embodiment of this application; Figure 4 A schematic diagram of a human face image provided in an embodiment of this application; Figure 5 A schematic diagram of five key facial features provided in an embodiment of this application; Figure 6 A flowchart illustrating a method for determining a specified identity type provided in an embodiment of this application; Figure 7 A flowchart illustrating a method for determining a specified identity type provided in an embodiment of this application; Figure 8 A flowchart illustrating a face recognition method provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a face recognition device provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0021] For ease of understanding, the terms used in the embodiments of this invention are explained below.

[0022] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0023] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0024] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to using cameras and computers to replace human eyes in recognizing and measuring targets, and then performing image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition. For example, in this embodiment, computer vision technology is used to perform facial recognition on a target object, and then the result of the facial recognition is used to determine whether the target object is a missing person.

[0025] Optimal frame: The face image with the best image quality in the face image sequence. The five key points of the face can be accurately located through this face image.

[0026] Five key facial features: key coordinates used to obtain user identity information.

[0027] It is understood that in the specific implementation of this application, data such as facial images and identity information are involved. When the following embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0028] The design concept of the embodiments of this application will be introduced below.

[0029] Currently, related technologies passively search for missing persons by posting missing persons information on websites. However, this approach has limited reach, with relatively few users seeing the posted missing persons information, resulting in low efficiency and success rates in searching for missing persons.

[0030] Analysis revealed that in real life, missing persons may have been missing for several years or even decades, or their living environment may have changed significantly before and after their disappearance. These factors can all lead to substantial changes in the appearance of missing persons. Furthermore, the reference facial images stored in facial databases are mostly images taken before the disappearance. In such cases, comparing the captured facial images with the reference images to determine the identity of the person is of low accuracy.

[0031] In view of this, this application provides a face recognition method, which includes: acquiring a face image of a target object to be recognized; extracting target face features from the face image to be recognized; and determining the target state dimension corresponding to the face image to be recognized based on the target face features. Then, based on the target face features and the target state dimension, predicting auxiliary face features corresponding to at least one auxiliary state dimension of the face image to be recognized. Finally, based on the target face features and the obtained auxiliary face features, determining whether the target object belongs to a specified identity type.

[0032] In this embodiment, target facial features in the target state dimension and auxiliary facial features in at least one auxiliary state dimension of the face image to be identified are extracted. Then, based on the target facial features and the obtained auxiliary facial features, it is determined whether the target object belongs to a specified identity type. Compared with facial features in a single state dimension, facial features in multiple state dimensions can more comprehensively represent different states of the face. Therefore, determining whether the target object belongs to a specified identity type based on facial features in multiple state dimensions can effectively improve the accuracy of face recognition, thereby improving the efficiency and success rate of searching for missing persons.

[0033] refer to Figure 1 This is a system architecture diagram applicable to the embodiments of this application. The system architecture includes at least a terminal device 101 and a server 102.

[0034] The terminal device 101 has an image acquisition function. It acquires a sequence of facial images and then selects facial images from the sequence whose image quality meets preset conditions for recognition. The terminal device 101 may include one or more processors 1011, a memory 1012, an I / O interface 1013 for interacting with the server 102, and a display panel 1014, etc. The terminal device 101 may be a payment device, a smart camera, a smartphone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smartwatch, etc., but is not limited to these.

[0035] Server 102 is the backend server for face recognition. Server 102 may include one or more processors 1021, memory 1022, and I / O interfaces 1023 for interacting with terminal device 101. Furthermore, server 102 may be configured with a database 1024. Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal device 101 and server 102 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0036] The face recognition method in this application embodiment can be executed by terminal device 101 or server 102.

[0037] In the first scenario, the facial recognition method can be executed by the terminal device 101.

[0038] Terminal device 101 acquires a sequence of face images and selects face images from the sequence whose image quality meets preset conditions for recognition. Then, it extracts the target face features from the face images to be recognized and determines the target state dimension corresponding to the face image based on these features. Next, based on the target face features and the target state dimension, it predicts auxiliary face features corresponding to at least one auxiliary state dimension of the face image to be recognized. Finally, based on the target face features and the obtained auxiliary face features, it determines whether the target object belongs to a specified identity type.

[0039] In the second scenario, the face recognition method can be executed by server 102.

[0040] Terminal device 101 acquires a sequence of face images, and then selects face images from the sequence whose image quality meets preset conditions for recognition. Terminal device 101 sends the face images to be recognized to server 102. Server 102 extracts the target face features from the face images to be recognized and determines the target state dimension corresponding to the face images based on these features. Then, based on the target face features and the target state dimension, it predicts the auxiliary face features corresponding to at least one auxiliary state dimension of the face images to be recognized. Finally, based on the target face features and the obtained auxiliary face features, it determines whether the target object belongs to a specified identity type.

[0041] based on Figure 1 The system architecture diagram shown in this application illustrates the flow of a face recognition method. Figure 2 As shown, the process of this method is as follows: Figure 1 The terminal device or server shown executes the following steps: Step 201: Obtain the face image of the target object to be identified.

[0042] Specifically, the target is people within the image capture range. Facial images captured by the terminal device may exhibit issues such as blurry images, incomplete faces, offset face positions, or faces appearing too small. For example, ... Figure 3 As shown, the face image only shows the left half of the face. For example, as... Figure 4 As shown, the face in the image is at a profile angle. Using these face images for face recognition will result in lower accuracy.

[0043] Therefore, in this embodiment, face image sequences are acquired in batches, and then face images in the sequence whose image quality meets a first preset condition are used as the face images to be identified. Specifically, the image quality of the face image includes image sharpness, face angle, face size, face centering, and whether it is a color image. The first preset condition can be set according to actual needs, such as the first preset condition being that the image sharpness is greater than a first threshold, the ratio of the face area to the total image area is greater than a second threshold, the face angle is frontal, the face position is centered, and the face image is a color image.

[0044] Different individuals correspond to different facial image sequences. For each individual, facial images whose quality meets a first preset condition are selected from the corresponding facial image sequence. If multiple facial images in a sequence meet the first preset condition, the optimal frame is selected from these multiple images as the facial image to be recognized. By selecting high-quality facial images from the facial image sequence for facial recognition, the accuracy of facial image recognition can be effectively improved.

[0045] Step 202: Extract the target face features from the face image to be identified, and determine the target state dimension corresponding to the face image to be identified based on the target face features.

[0046] Specifically, the target facial features include facial landmarks. For example, such as... Figure 5 As shown, by analyzing the face image to be recognized, five key points of the face are obtained: key points 501, 502, 503, 504, and 505. Key points 501 and 502 are key points above the eyes, key point 503 is a key point above the nose, and key points 504 and 505 are key points above the mouth. To achieve higher precision in face localization, the facial key points can be further divided into internal key points and contour key points. The internal key points include key points for the eyebrows, eyes, nose, and mouth.

[0047] The state dimension is used to characterize the state of a face in the image to be identified. The state of the face includes at least one of the following: face age, face emotion, face weight (weight or thickness), and degree of face modification. The target state dimension refers to the current state of the face in the image to be identified. For example, the current age of the face in the image to be identified might be between 25 and 30 years old; the current weight of the face in the image to be identified might be described as "overweight"; or the current emotion of the face in the image to be identified might be described as "angry."

[0048] Step 203: Based on the target face features and the target state dimension, predict the auxiliary face features corresponding to at least one auxiliary state dimension of the face image to be identified.

[0049] Specifically, the auxiliary state dimension is a state dimension that is different from the target state dimension.

[0050] One possible implementation involves pre-setting a target association relationship between at least one auxiliary state dimension and the target state dimension. After determining the target state dimension corresponding to the face image to be identified, at least one auxiliary state dimension is determined based on the target association relationship. Then, based on the target face features, auxiliary face features corresponding to the face image to be identified in at least one auxiliary state dimension are predicted.

[0051] For example, the target state dimension is set as "face age 25 years old". Two auxiliary state dimensions are pre-defined with a target association relationship of 20 years between them and the target state dimension. Based on the target state dimension (face age 25 years old) and the aforementioned target association relationship, the two auxiliary state dimensions are "face age 5 years old" and "face age 45 years old". Then, based on the target face features corresponding to age 25, auxiliary face features of the face in the image to be recognized at ages 5 and 45 are pre-defined.

[0052] One possible implementation involves pre-setting at least one auxiliary state dimension. After extracting the target face features from the face image to be recognized, it is first determined whether the target state dimension exists in the at least one auxiliary state dimension. If it exists, the target state dimension is removed from the at least one auxiliary state dimension. Then, based on the target face features, the auxiliary face features corresponding to the face image to be recognized in other auxiliary state dimensions are predicted. Otherwise, based on the target face features, the auxiliary face features corresponding to the face image to be recognized in the at least one auxiliary state dimension are predicted.

[0053] For example, the target state dimension is set as "face age 25 years old", and two auxiliary state dimensions are preset as "face age 25 years old" and "face age 35 years old". Since the target state dimension "face age 25 years old" exists in the auxiliary state dimensions, it is not necessary to predict the auxiliary face features in the case of "face age 25 years old". It is only necessary to preset the auxiliary face features of the face in the image to be identified in the case of "face age 35 years old" based on the target face features of "face age 25 years old".

[0054] Optionally, a neural network model is employed to predict auxiliary facial features corresponding to at least one auxiliary state dimension of the target facial image, based on the target facial features. Specifically, the neural network model can be a Convolutional Neural Network (CNN), a Deep Neural Network (DNN), or the like. Optionally, in this embodiment, the trained neural network model can be stored on a blockchain.

[0055] Taking age as the state dimension as an example, facial images of people at different ages are collected in advance as training samples to train a convolutional neural network. After training, the convolutional neural network learns the feature relationships between facial features across different age dimensions. Then, based on the target facial features, the convolutional neural network predicts auxiliary facial features corresponding to at least one age state dimension of the face image to be identified.

[0056] Step 204: Based on the target face features and the obtained auxiliary face features, determine whether the target object is of the specified identity type.

[0057] Specifically, both the target facial features and the obtained auxiliary facial features refer to the same person. The extracted target facial features and the obtained auxiliary facial features are stored in an information queue, and the facial feature data in the information queue is processed one by one. Specifically, the target facial features and the obtained auxiliary facial features are compared with the facial features of reference facial images in a facial image database to determine whether the target object belongs to the specified identity type.

[0058] Each reference face image in the face image database corresponds to a unique identity information, including name, gender, ID card information, mobile phone number, and identity type.

[0059] The specified identity type can be set according to actual needs. Taking the specified identity type as missing persons as an example, a facial image database corresponding to missing persons is set up in advance. The facial features of the target person and the obtained auxiliary facial features are compared with the facial features of the missing persons' facial images in the facial image database to determine whether the target person is a missing person.

[0060] In this embodiment, target facial features in the target state dimension and auxiliary facial features in at least one auxiliary state dimension of the face image to be identified are extracted. Then, based on the target facial features and the obtained auxiliary facial features, it is determined whether the target object belongs to a specified identity type. Compared with facial features in a single state dimension, facial features in multiple state dimensions can more comprehensively represent different states of the face. Therefore, determining whether the target object belongs to a specified identity type based on facial features in multiple state dimensions can effectively improve the accuracy of face recognition, thereby improving the efficiency and success rate of searching for missing persons.

[0061] Optionally, when the target is determined to be a missing person, the target acquisition location and target acquisition time of the face image to be identified are obtained.

[0062] Specifically, the target acquisition location is the location of the terminal device that acquires the face image to be identified, and the target acquisition time is the time when the terminal device acquires the face image to be identified. Simultaneously, the identity information of the face image to be identified can be obtained from a face image database. Then, based on the face image to be identified, its identity information, the target acquisition location, and the target acquisition time, the location or movement trajectory of a missing person can be quickly determined, enabling faster recovery of the missing person.

[0063] Optionally, regarding step 204 above, based on the target facial features and the obtained auxiliary facial features, to determine whether the target object belongs to the specified identity type, this application embodiment provides at least the following implementation methods: Implementation Method 1: If at least one of the target facial features and the obtained auxiliary facial features matches a facial feature of a first reference facial image in the first facial image database, then the target object is determined to be of the specified identity type, wherein the first reference facial image is the facial image corresponding to the specified identity type. If none of the target facial features and the obtained auxiliary facial features match a facial feature of the first reference facial image in the first facial image database, then the target object is determined not to be of the specified identity type.

[0064] Specifically, the first face image library includes at least one first reference face image, each first reference face image corresponding to an identity information, and the identity type of all first reference face images in the first face image library is a specified identity type. The facial features of the first reference face image are obtained after pre-extracting features from the first reference face image, and the facial features of the first reference face image include facial features of the first reference face image in one or more state dimensions.

[0065] In practice, a method can be used to first compare the target facial features and then compare the auxiliary facial features to determine whether the target object belongs to the specified identity type. Figure 6 As shown, it includes the following steps: Step 601: Compare the target face features with the face features of each first reference face image in the first face image library.

[0066] Step 602: Determine whether there are facial features in each first reference face image that match the target face features. If yes, proceed to step 603; otherwise, proceed to step 604.

[0067] In practice, for a first reference face image, the similarity between the face features of the first reference face image and the target face features is calculated. If the similarity is greater than a preset threshold, the face features of the first reference face image match the target face features; otherwise, the face features of the first reference face image do not match the target face features.

[0068] Step 603: Determine the target object as a specified identity type, and use the identity information of the first matching reference face image as the identity information of the face image to be identified.

[0069] Step 604: Compare the auxiliary facial features with the facial features of each first reference facial image in the first facial image library.

[0070] Step 605: Determine whether there are facial features in each first reference face image that match the auxiliary face features. If yes, proceed to step 606; otherwise, proceed to step 607.

[0071] For a first reference face image, calculate the similarity between the face features of the first reference face image and the auxiliary face features. If the similarity is greater than a preset threshold, the face features of the first reference face image match the auxiliary face features; otherwise, the face features of the first reference face image do not match the auxiliary face features.

[0072] Step 606: Determine the target object as a specified identity type, and use the identity information of the first matching reference face image as the identity information of the face image to be identified.

[0073] Step 607: Determine that the target object is not of the specified identity type.

[0074] It should be noted that in this embodiment, there may be one or more auxiliary facial features. When comparing multiple auxiliary facial features, they can be compared with the first facial database sequentially in a preset order, or they can be compared randomly. Alternatively, in this embodiment, the auxiliary facial features can be compared first, followed by the target facial features, to determine whether the target object belongs to a specified identity type; this will not be elaborated upon here.

[0075] In this embodiment, if at least one of the target facial features and auxiliary facial features matches the facial features of the first reference facial image, the target object is determined to be of the specified identity type. Therefore, when the target facial feature fails to match the facial features of the first reference facial image because they are not facial features of the same state dimension, the auxiliary facial feature can still successfully match the facial features of the first reference facial image, thereby improving the accuracy of facial recognition and reducing the false negative rate when searching for missing persons.

[0076] Implementation Method 2: Obtain the facial depth image of the target object and extract the facial depth features from the facial depth image. Based on the target facial features and facial depth features, obtain the comprehensive facial features of the target object. Then, based on the comprehensive facial features and the obtained auxiliary facial features, determine whether the target object belongs to the specified identity type.

[0077] Specifically, a depth camera in a terminal device captures infrared light from a mottled structure, and then analyzes this infrared light to obtain a facial depth image. The facial depth image includes information related to the distance between the depth camera and the facial surface. Each pixel in the facial depth image represents the vertical distance between the depth camera and the facial surface, typically represented by 16 bits in millimeters. The target facial features are generally two-dimensional facial landmarks; based on these two-dimensional landmarks and facial depth features, a comprehensive three-dimensional facial feature can be obtained.

[0078] Sometimes, the faces in depth images captured by a depth camera may be incomplete, such as only half of a face. In this case, combining facial landmarks with facial depth features will result in incomplete composite facial features. Using incomplete composite facial features for face recognition will affect the accuracy of face recognition. Therefore, in this embodiment, a depth image sequence of the target object is acquired in batches using a depth camera, and then the depth images in the sequence whose completeness meets a second preset condition are used as the face depth images.

[0079] Optionally, when the face depth image and the face image to be identified correspond to the same face at the same time, combining the face key points with the face depth features will result in a comprehensive face feature that better represents the face information. Therefore, when selecting the face image to be identified from the face image sequence based on image quality, and when selecting the face depth image from the depth image sequence based on completeness, it is necessary to simultaneously satisfy the condition that the face image to be identified and the face depth image were acquired at the same time.

[0080] The facial features of the first reference face image are obtained by pre-extracting features from the first reference face image. Specifically, two-dimensional facial features of the first reference face image in one or more state dimensions are first extracted, and then combined with the facial depth features of the facial depth image corresponding to the face in the first reference face image to obtain three-dimensional reference face features of the first reference face image in one or more state dimensions. The facial features of the first reference face image may include only two-dimensional facial features of the first reference face image in one or more state dimensions, or only three-dimensional reference face features of the first reference face image in one or more state dimensions, or both two-dimensional and three-dimensional reference face features.

[0081] The integrated facial features and the obtained auxiliary facial features are compared with the facial features of the first reference facial image in the first facial image library to determine whether the target object is of the specified identity type.

[0082] If, among the combined facial features and the obtained auxiliary facial features, at least one facial feature matches the facial features of the first reference facial image in the first facial image library, then the target object is determined to be of the specified identity type, wherein the first reference facial image is the facial image corresponding to the specified identity type. If, among the combined facial features and the obtained auxiliary facial features, no facial feature matches the facial features of the first reference facial image in the first facial image library, then the target object is determined not to be of the specified identity type.

[0083] In specific comparisons, a method can be used to first compare comprehensive facial features, and then compare auxiliary facial features to determine whether the target object belongs to the specified identity type. Specifically, for example... Figure 7 As shown, it includes the following steps: Step 701: Compare the integrated facial features with the facial features of each first reference facial image in the first facial image library.

[0084] Step 702: Determine whether there are facial features in each first reference face image that match the comprehensive face features. If so, proceed to step 703; otherwise, proceed to step 704.

[0085] In practice, for a first reference face image, the similarity between the face features of the first reference face image and the comprehensive face features is calculated. If the similarity is greater than a preset threshold, the face features of the first reference face image match the comprehensive face features; otherwise, the face features of the first reference face image do not match the comprehensive face features.

[0086] Step 703: Determine the target object as a specified identity type, and use the identity information of the first matching reference face image as the identity information of the face image to be identified.

[0087] Step 704: Compare the auxiliary facial features with the facial features of each first reference facial image in the first facial image library.

[0088] Step 705: Determine whether there are facial features in each first reference face image that match the auxiliary face features. If yes, proceed to step 706; otherwise, proceed to step 707.

[0089] For a first reference face image, calculate the similarity between the face features of the first reference face image and the auxiliary face features. If the similarity is greater than a preset threshold, the face features of the first reference face image match the auxiliary face features; otherwise, the face features of the first reference face image do not match the auxiliary face features.

[0090] Step 706: Determine the target object as a specified identity type, and use the identity information of the first matching reference face image as the identity information of the face image to be identified.

[0091] Step 707: Determine that the target object is not of the specified identity type.

[0092] It should be noted that in this embodiment, there may be one or more auxiliary facial features. When comparing multiple auxiliary facial features, they can be compared with the first facial database sequentially in a preset order, or they can be compared randomly. Alternatively, in this embodiment, the auxiliary facial features can be compared first, followed by the combined facial features, to determine whether the target object belongs to a specified identity type; this will not be elaborated upon here.

[0093] In this embodiment, a three-dimensional comprehensive facial feature is obtained based on facial key points and facial depth features. Since the three-dimensional comprehensive facial feature can more comprehensively represent the face, comparing the comprehensive facial feature and auxiliary facial features with the facial features of the first reference facial image to determine whether the target object belongs to the specified identity type can effectively improve the accuracy of facial recognition.

[0094] Implementation Method 3: Acquire a depth image of the target object's face and extract its depth features. Based on the target face features and depth features, obtain the comprehensive face features of the target object. Acquire an infrared image of the target object's face and extract its infrared features. When the infrared features meet the liveness detection criteria, determine whether the target object belongs to the specified identity type based on the comprehensive face features and the obtained auxiliary face features.

[0095] Specifically, the facial infrared image is obtained by capturing infrared light images through an infrared camera in the terminal device. The facial infrared image can be used for liveness detection to ensure that the target object is a real person, not a photograph or model, etc., which are not living entities. Furthermore, to ensure the accuracy of liveness detection, in this embodiment, an infrared camera is used to batch-collect infrared image sequences of the target object, and then the depth images in the infrared image sequence whose brightness meets a third preset condition are used as the facial infrared images.

[0096] When the infrared facial features meet the liveness detection criteria, the integrated facial features and the obtained auxiliary facial features are compared with the facial features of the first reference facial image in the first facial image library to determine whether the target object is of the specified identity type.

[0097] If at least one facial feature among the combined facial features and the obtained auxiliary facial features matches a facial feature of a first reference facial image in the first facial image database, then the target object is determined to be of the specified identity type, where the first reference facial image is the facial image corresponding to the specified identity type. If none of the combined facial features and the obtained auxiliary facial features match a facial feature of the first reference facial image in the first facial image database, then the target object is determined not to be of the specified identity type.

[0098] In this application, a three-dimensional comprehensive facial feature is obtained based on facial key points and facial depth features. The comprehensive facial feature and auxiliary facial feature are compared with the facial features of the first reference facial image. At the same time, with the assistance of facial infrared features, it is ensured that the target object is a real living person. The three work together to ensure that the facial recognition has higher accuracy and security.

[0099] Optionally, in the three embodiments described above, compared to missing persons, the facial images of non-missing persons in the facial database are less different from their current appearance. Therefore, when performing facial recognition on non-missing persons based on reference facial images of them in the facial image database, fewer facial features can be relied upon while ensuring accuracy. Furthermore, facial images of non-missing persons are easier to obtain. Therefore, the facial image to be identified is first compared with the reference facial image of the non-missing person; if there is a mismatch, it is compared with the reference facial image of the missing person; otherwise, it is not necessary to compare with the reference facial image of the missing person. This improves the efficiency of facial recognition while reducing resource consumption.

[0100] Therefore, in this embodiment of the application, before determining the target state dimension corresponding to the face image to be identified based on the target face features, it is first determined that the target face features do not match the face features of the second reference face image in the second face image library, wherein the second reference face image is a face image corresponding to a non-specified identity type.

[0101] Specifically, the second face image library includes at least one second reference face image, each second reference face image corresponding to an identity information, and the identity type of all second reference face images in the second face image library is a non-specified identity type. The facial features of the second reference face image are obtained after pre-extracting features from the second reference face image, and the facial features of the second reference face image include facial features of the second reference face image in one or more state dimensions.

[0102] If the target face features do not match the face features of the second reference face images, it indicates that the target object's identity type is not a non-specified identity type. In this case, it is necessary to further extract auxiliary face features corresponding to at least one auxiliary state dimension of the face image to be identified, and determine whether the target object belongs to the specified identity type based on the target face features and the obtained auxiliary face features. Conversely, if it is determined that there are face features in each of the second reference face images that match the target face features, it indicates that the target object's identity type is a non-specified identity type, and the subsequent steps of extracting auxiliary face features and comparison are not required.

[0103] In this embodiment, auxiliary facial features of the face image to be identified are extracted only when the facial features of the target face do not match those of the face image corresponding to a non-specified identity type. Based on the target face features and the obtained auxiliary facial features, it is determined whether the target object is of the specified identity type. Otherwise, the subsequent extraction and comparison of auxiliary facial features can be skipped, thereby improving the efficiency of face recognition and reducing the consumption of computing resources.

[0104] Implementation method four involves setting up a third-party face image library and an identity information library corresponding to a specified identity type. The third-party face image library and the identity information library corresponding to the specified identity type can be located on the same server or on different servers. The third-party face image library includes at least one third-party reference face image, and each third-party reference face image corresponds to one identity information, which does not include the identity type. The facial features of the third-party reference face image are obtained by pre-extracting features from the third-party reference face image, and the facial features of the third-party reference face image include facial features of the third-party reference face image in one or more state dimensions.

[0105] The target facial features and the obtained auxiliary facial features are compared with facial images in a third facial image database. If at least one facial feature in either the target facial features or the obtained auxiliary facial features matches a facial feature in a third reference facial image, the identity information of the matching third reference facial image is used as the identity information of the target object. Then, the identity information of the target object is compared with the identity information database corresponding to the specified identity type. If a matching identity information exists in the identity information database, the target object is determined to be of the specified identity type.

[0106] In this embodiment, a face image database and an identity information database are set up respectively. First, the identity information of the target object is determined by comparing the target face features and auxiliary face features with the face image database. Then, the identity information of the target object is compared with the identity information database to determine whether the target object belongs to the specified identity type. This improves the efficiency of finding missing persons and reduces the risk of information leakage.

[0107] To better explain the embodiments of this application, the following uses a missing persons investigation scenario as an example to introduce a face recognition method provided by the embodiments of this application. This method is executed interactively by a terminal device and a face recognition server, such as... Figure 8 As shown, it includes the following steps: The terminal device acquires a sequence of facial images. During acquisition, it uses a depth camera and an infrared camera to capture depth and infrared image sequences of the person. The facial, depth, and infrared images acquired simultaneously are combined into a single facial image set. Then, based on the image quality of the facial images, the completeness of the depth images, and the depth of the infrared images, each facial image set is filtered to obtain the optimal set, which is stored in the optimal frame queue. At preset time intervals, the terminal device checks if the optimal frame queue contains the optimal facial image set. If so, it converts the optimal facial image set into a byte stream and sends the byte stream to the facial recognition server.

[0108] The face recognition server receives the optimal set of face images uploaded by various terminal devices. This optimal set includes the face image to be recognized, a face depth image, and a face infrared image. The server analyzes the face image to obtain the five key facial features (Five Points) and determines the target state dimension. Based on the Five Keys and the target state dimension, it predicts auxiliary face features corresponding to at least one auxiliary state dimension. The server also analyzes the face depth image to obtain face depth features and the face infrared image to obtain face infrared features. These features—Five Keys, auxiliary face features, face depth features, and face infrared features—are then stored as a set of face features in an information queue. The face recognition server processes each set of face features in the information queue sequentially.

[0109] Specifically, for a target person, a set of facial features is defined, including five key points of the target face, auxiliary facial features, depth features, and infrared features. The face recognition server obtains a comprehensive three-dimensional facial feature based on the five key points and depth features. The infrared features determine if the target is a live person. If so, the comprehensive facial feature and the obtained auxiliary facial feature are compared with the facial features of a reference facial image in the face image database. If at least one facial feature from the comprehensive facial feature and the obtained auxiliary facial feature matches the facial features of a reference facial image in the face image database, the identity information of the matching reference facial image is used as the identity information of the target person.

[0110] The facial recognition server obtains the identity information of missing persons through the interface provided by the population detection server, and then compares the identity information of the target person with the obtained identity information of the missing persons. If there is a target person's identity information that matches the target person's identity information in the obtained missing persons information, the target person is determined to be a missing person.

[0111] The facial recognition server acquires environmental data of the target person. This environmental data includes the target acquisition location and time of the facial image to be identified, corresponding to the aforementioned set of facial features. The facial image to be identified, the target acquisition location, the target acquisition time, and the target person's identity information are then packaged into a JSON file. This JSON file is then sent to a pre-defined interface / email address for rapid location of missing persons.

[0112] In this embodiment, target facial features in the target state dimension and auxiliary facial features in at least one auxiliary state dimension of the face image to be identified are extracted. Then, based on the target facial features and the obtained auxiliary facial features, it is determined whether the target object belongs to a specified identity type. Compared with facial features in a single state dimension, facial features in multiple state dimensions can more comprehensively represent different states of the face. Therefore, determining whether the target object belongs to a specified identity type based on facial features in multiple state dimensions can effectively improve the accuracy of face recognition, thereby improving the efficiency and success rate of searching for missing persons.

[0113] Based on the same technical concept, embodiments of this application provide a face recognition device, such as... Figure 9 As shown, the device 900 includes: The acquisition module 901 is used to acquire the face image of the target object to be identified; The feature extraction module 902 is used to extract the target face features of the face image to be identified, and determine the target state dimension corresponding to the face image to be identified based on the target face features; The feature expansion module 903 is used to predict the auxiliary face features corresponding to at least one auxiliary state dimension of the face image to be identified based on the target face features and the target state dimension. The discrimination module 904 is used to determine whether the target object is of a specified identity type based on the target face features and the obtained auxiliary face features.

[0114] Optionally, the discrimination module 904 is specifically used for: Obtain the facial depth image of the target object, and extract the facial depth features from the facial depth image; Based on the target facial features and the facial depth features, the comprehensive facial features of the target object are obtained; Based on the comprehensive facial features and the obtained auxiliary facial features, it is determined whether the target object belongs to the specified identity type.

[0115] Optionally, the discrimination module 904 is specifically used for: If at least one of the comprehensive facial features and the obtained auxiliary facial features matches the facial features of the first reference facial image in the first facial image library, then the target object is determined to be of a specified identity type, wherein the first reference facial image is the facial image corresponding to the specified identity type. If none of the integrated facial features and the obtained auxiliary facial features match the facial features of the first reference facial image in the first facial image library, then the target object is determined not to be of the specified identity type.

[0116] Optionally, the discrimination module 904 is further configured to: Before determining whether the target object is of a specified identity type based on the comprehensive facial features and the obtained auxiliary facial features, the infrared image of the target object's face is acquired; Extract the facial infrared features from the facial infrared image and determine that the facial infrared features meet the conditions for liveness detection.

[0117] Optionally, the feature extraction module 902 is further configured to: Before extracting the target face features from the face image to be identified, it is determined that the image quality of the face image to be identified meets a first preset condition; Before extracting the facial depth features from the facial depth image, it is determined that the integrity of the facial depth image meets a second preset condition. Before extracting the facial infrared features from the facial infrared image, it is determined that the brightness of the facial infrared image meets a third preset condition.

[0118] Optionally, the discrimination module 904 is specifically used for: If at least one of the target facial features and the obtained auxiliary facial features matches the facial features of the first reference facial image in the first facial image library, then the target object is determined to be of a specified identity type, wherein the first reference facial image is the facial image corresponding to the specified identity type. If none of the target facial features and the obtained auxiliary facial features match the facial features of the first reference facial image in the first facial image library, then the target object is determined not to be of the specified identity type.

[0119] Optionally, the discrimination module 904 is further configured to: Before determining the target state dimension corresponding to the face image to be identified based on the target face features, it is determined that the target face features do not match the face features of the second reference face image in the second face image library, wherein the second reference face image is a face image corresponding to a non-specified identity type.

[0120] Optionally, the specified identity type is missing persons; Optionally, the discrimination module 904 is further configured to: After determining that the target object is of a specified identity type, the target acquisition location and target acquisition time of the face image to be identified are obtained.

[0121] In this embodiment, target facial features in the target state dimension and auxiliary facial features in at least one auxiliary state dimension of the face image to be identified are extracted. Then, based on the target facial features and the obtained auxiliary facial features, it is determined whether the target object belongs to a specified identity type. Compared with facial features in a single state dimension, facial features in multiple state dimensions can more comprehensively represent different states of the face. Therefore, determining whether the target object belongs to a specified identity type based on facial features in multiple state dimensions can effectively improve the accuracy of face recognition, thereby improving the efficiency and success rate of searching for missing persons.

[0122] Based on the same technical concept, embodiments of this application provide a computer device, such as... Figure 10 As shown, it includes at least one processor 1001 and a memory 1002 connected to at least one processor. In this embodiment, the specific connection medium between the processor 1001 and the memory 1002 is not limited. Figure 10 Taking the connection between processor 1001 and memory 1002 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0123] In this embodiment of the application, the memory 1002 stores instructions that can be executed by at least one processor 1001. By executing the instructions stored in the memory 1002, at least one processor 1001 can perform the steps of the aforementioned face recognition method.

[0124] The processor 1001 is the control center of the computer device, capable of connecting to various parts of the device via various interfaces and lines. It performs facial recognition by running or executing instructions stored in the memory 1002 and accessing data stored in the memory 1002. Optionally, the processor 1001 may include one or more processing units. The processor 1001 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1001. In some embodiments, the processor 1001 and the memory 1002 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0125] The processor 1001 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0126] Memory 1002, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 1002 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory 1002 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 1002 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0127] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described face recognition method.

[0128] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0130] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0131] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0132] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0133] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A face recognition method, characterized in that, include: Acquire the image of the face to be identified from the target object; Extract the target facial features from the face image to be identified; When it is determined that there is a face feature matching the target face feature among the face features of each second reference face image contained in the second face image library, the identity type of the target object is determined to be an unspecified identity type, the second reference face image is the face image corresponding to the unspecified identity type, and the unspecified identity type is a non-missing person; the face features of the second reference face image include the face features of the second reference face image in multiple state dimensions; When it is determined that the target face feature does not match the face features of each second reference face image, the target state dimension corresponding to the face image to be identified is determined based on the target face feature. The target state dimension refers to the current state of the face in the face image to be identified. The current state of the face includes: face age, face emotion, face fatness / thinness, and face modification degree. Based on the target state dimension and the preset target association relationship, at least one auxiliary state dimension is determined, wherein the auxiliary state dimension is a state dimension different from the target state dimension. A neural network model is used to predict the auxiliary facial features of the target facial image in at least one auxiliary state dimension based on the target facial features; the neural network model learns through training to obtain the feature association relationship between facial features in different state dimensions; Extract facial depth features from the facial depth image, and obtain the comprehensive three-dimensional facial features of the target object based on the target facial features and the facial depth features; If at least one of the integrated facial features and the obtained auxiliary facial features matches the facial features of the first reference facial image in the first facial image library, then the target object is determined to be of the specified identity type.

2. The method as described in claim 1, characterized in that, Also includes: If none of the integrated facial features and the obtained auxiliary facial features match the facial features of the first reference facial image in the first facial image library, then the target object is determined not to be of the specified identity type.

3. The method as described in claim 1, characterized in that, Also includes: Acquire the infrared image of the target object's face; Extract the facial infrared features from the facial infrared image and determine that the facial infrared features meet the conditions for liveness detection.

4. The method as described in claim 3, characterized in that, Before extracting the target facial features from the face image to be identified, the process further includes: Determine that the image quality of the face image to be identified meets a first preset condition; Before extracting the facial depth features from the facial depth image, the process further includes: Determine that the integrity of the facial depth image meets the second preset condition; Before extracting the facial infrared features from the facial infrared image, the process also includes: The brightness of the infrared image of the face is determined to meet the third preset condition.

5. The method as described in any one of claims 1 to 4, characterized in that, The specified identity type is missing persons; After determining that the target object is of a specified identity type, the process further includes: Obtain the target acquisition location and target acquisition time of the face image to be identified.

6. A face recognition device, characterized in that, include: The acquisition module is used to acquire the face image of the target object to be identified; The feature extraction module is used to extract the target facial features from the face image to be identified; When it is determined that there is a face feature matching the target face feature among the face features of each second reference face image contained in the second face image library, the identity type of the target object is determined to be an unspecified identity type, the second reference face image is the face image corresponding to the unspecified identity type, and the unspecified identity type is a non-missing person; the face features of the second reference face image include the face features of the second reference face image in multiple state dimensions; When it is determined that the target face feature does not match the face features of each second reference face image, the target state dimension corresponding to the face image to be identified is determined based on the target face feature. The target state dimension refers to the current state of the face in the face image to be identified. The current state of the face includes: face age, face emotion, face fatness / thinness, and face modification degree. The feature expansion module is used to determine at least one auxiliary state dimension based on the target state dimension and a preset target association relationship, wherein the auxiliary state dimension is a state dimension different from the target state dimension; and to use a neural network model to predict the auxiliary face features corresponding to the face image to be identified in at least one auxiliary state dimension based on the target face features; the neural network model learns through training to obtain the feature association relationship between face features in different state dimensions; The discrimination module is used to extract facial depth features from the facial depth image and obtain the three-dimensional comprehensive facial features of the target object based on the target facial features and the facial depth features. If at least one of the integrated facial features and the obtained auxiliary facial features matches the facial features of the first reference facial image in the first facial image library, then the target object is determined to be of the specified identity type.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the method according to any one of claims 1 to 5.

Images