A face recognition method, device and equipment
By performing coordinate conversion and fitting processing on the 3D face map, 3D parameterized face data is generated, which solves the recognition problem of 3D face recognition in an unstable environment, improves recognition accuracy and privacy protection, and reduces storage Space demand.
Patent Information
- Application Number
- CN202210181933.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-25
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-02-25
AI Technical Summary
The existing 3D face recognition solution has poor imaging effects when the user distance exceeds the camera range in a strong light environment or when the user distance exceeds the camera range, resulting in the impact of recognition stability and making it difficult to achieve accurate face recognition.
By collecting 3D face maps, coordinate conversion is performed to obtain 3D face point clouds, using pre-trained fitting parameter prediction model to predict fit parameters, fit the preset 3D reference face grid, and obtain 3D parameterization Face data, so as to be recognized.
It improves the accuracy and reliability of 3D face recognition, reduces the impact of 3D camera instability on the algorithm, and reduces the storage space requirements through parameterized processing, and enhances privacy protection.
Smart Images

Figure CN114511911B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method, device and equipment for face recognition. Background Art
[0002] With the development of computer and Internet technology, many businesses use facial recognition to authenticate users, such as face payment and face access control.
[0003] Face recognition usually includes two forms, 2D face recognition and 3D face recognition. For 3D face recognition, a 3D camera is required to collect the user's 3D face image and perform face recognition on the user through the 3D face image. There are many types of 3D cameras, such as structured light 3D cameras, ToF 3D cameras, etc. Taking the widely used structured light 3D camera as an example, in some special scenarios (for example, strong light environment, when the user is out of range from the camera), the imaging effect of the 3D face image collected by the structured light 3D camera is often poor.
[0004] Based on this, a more reliable face recognition solution is needed. Summary of the invention
[0005] One or more embodiments of the present specification provide a face recognition method, apparatus, device, and storage medium to solve the following technical problem: a more reliable face recognition solution is needed.
[0006] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:
[0007] One or more embodiments of this specification provide a face recognition method, including:
[0008] After the user's face is in a predetermined face placement position, collecting a 3D face image based on the depth information of the user's face;
[0009] Performing coordinate transformation on the 3D face image to obtain a 3D face point cloud corresponding to the user's face;
[0010] According to the 3D face point cloud, the fitting parameters required for 3D face reconstruction are predicted by a pre-trained fitting parameter prediction model;
[0011] According to the fitting parameters, a preset 3D reference face mesh is fitted to obtain 3D parameterized face data, wherein the 3D reference face mesh includes a plurality of vertices, the plurality of vertices are connected to each other, and each of the vertices is respectively provided with face semantics;
[0012] The user is identified according to the 3D parameterized face data.
[0013] One or more embodiments of this specification provide a face recognition device, including:
[0014] A collection module, after the user's face is in a predetermined face placement position, collects a 3D face image based on the depth information of the user's face;
[0015] A coordinate conversion module performs coordinate conversion on the 3D face image to obtain a 3D face point cloud corresponding to the user's face;
[0016] A fitting parameter prediction module, which predicts the fitting parameters required for 3D face reconstruction based on the 3D face point cloud using a pre-trained fitting parameter prediction model;
[0017] A parameterization module, which performs fitting processing on a preset 3D reference face mesh according to the fitting parameters to obtain 3D parameterized face data, wherein the 3D reference face mesh includes a plurality of vertices, the plurality of vertices are connected to each other, and each of the vertices is respectively provided with face semantics;
[0018] A recognition module is used to recognize the user according to the 3D parameterized face data.
[0019] One or more embodiments of this specification provide a face recognition device, including:
[0020] at least one processor; and,
[0021] a memory communicatively connected to the at least one processor; wherein,
[0022] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:
[0023] After the user's face is in a predetermined face placement position, collecting a 3D face image based on the depth information of the user's face;
[0024] Performing coordinate transformation on the 3D face image to obtain a 3D face point cloud corresponding to the user's face;
[0025] According to the 3D face point cloud, the fitting parameters required for 3D face reconstruction are predicted by a pre-trained fitting parameter prediction model;
[0026] According to the fitting parameters, a preset 3D reference face mesh is fitted to obtain 3D parameterized face data, wherein the 3D reference face mesh includes a plurality of vertices, the plurality of vertices are connected to each other, and each of the vertices is respectively provided with face semantics;
[0027] The user is identified according to the 3D parameterized face data.
[0028] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to:
[0029] After the user's face is in a predetermined face placement position, collecting a 3D face image based on the depth information of the user's face;
[0030] Performing coordinate transformation on the 3D face image to obtain a 3D face point cloud corresponding to the user's face;
[0031] According to the 3D face point cloud, the fitting parameters required for 3D face reconstruction are predicted by a pre-trained fitting parameter prediction model;
[0032] According to the fitting parameters, a preset 3D reference face mesh is fitted to obtain 3D parameterized face data, wherein the 3D reference face mesh includes a plurality of vertices, the plurality of vertices are connected to each other, and each of the vertices is respectively provided with face semantics;
[0033] The user is identified according to the 3D parameterized face data.
[0034] At least one of the above technical solutions adopted in one or more embodiments of this specification can achieve the following beneficial effects:
[0035] Compared with 2D face images, face recognition through 3D face images is very user-friendly and can ensure the privacy and security needs of users. By pre-parameterizing the 3D reference face grid and combining the predicted fitting parameters, the parameterization of the user's 3D face image can be achieved, which can reduce the impact of the instability of the 3D camera on the face recognition algorithm. In addition, since the face data is parameterized, the backup system does not need to store the feature summary of the data, and can directly store the parameterized 3D face data, effectively reducing the system storage space. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0037] Figure 1A flowchart of a face recognition method provided for one or more embodiments of this specification;
[0038] Figure 2 A schematic diagram of a 2D face image in an application scenario provided for one or more embodiments of this specification;
[0039] Figure 3 A schematic diagram of a 3D face image in an application scenario provided for one or more embodiments of this specification;
[0040] Figure 4 A 3D face image with poor imaging effect in an application scenario provided for one or more embodiments of this specification;
[0041] Figure 5 A schematic diagram of coordinate conversion in an application scenario provided for one or more embodiments of this specification;
[0042] Figure 6 A schematic diagram of target detection and key point detection in an application scenario provided for one or more embodiments of this specification;
[0043] Figure 7 A schematic diagram of a process for implementing 3D parameterized face data in an application scenario provided for one or more embodiments of this specification;
[0044] Figure 8 A schematic diagram of 3D parameterized face data in an application scenario provided for one or more embodiments of this specification;
[0045] Fig. 9 A schematic diagram of face similarity confirmation in an application scenario provided for one or more embodiments of this specification;
[0046] Fig.10 A flowchart of a face recognition method in an application scenario provided for one or more embodiments of this specification;
[0047] Fig.11 A schematic diagram of the structure of a face recognition device provided for one or more embodiments of this specification;
[0048] Fig.12 A schematic diagram of the structure of a face recognition device provided for one or more embodiments of this specification. DETAILED DESCRIPTION
[0049] The embodiments of this specification provide a face recognition method, apparatus, device and storage medium.
[0050] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.
[0051] Figure 1 A flow chart of a face recognition method provided for one or more embodiments of this specification. The method can be applied to different business fields, such as offline automatic payment, Internet financial business, e-commerce business, instant messaging business, game business, official business, etc. The process can be executed by a computing device in the corresponding field (such as a face payment machine for offline payment), and some input parameters or intermediate results in the process allow manual intervention and adjustment to help improve accuracy.
[0052] In order to be able to Figure 1 The scheme shown is explained, and face recognition in the traditional scheme is first introduced here. Figure 2 and Figure 3 Schematic diagrams of 2D face images and 3D face images in an application scenario are provided respectively for one or more embodiments of the present specification. As can be seen from the figure, compared with 2D face images, 3D face images are less recognizable to the naked eye, and 3D face recognition has obvious advantages over 2D face recognition in terms of privacy protection.
[0053] For 3D face recognition, most of the 3D cameras currently installed in face scanning devices are relatively low-cost structured light 3D cameras. Such cameras usually collect 3D face images based on depth information. The 3D face images store face data in the form of 2D images, and the corresponding depth information is stored for each pixel in the 2D image. In traditional 3D face recognition, the face identity is identified based on algorithmic analysis of the 3D face image. The defect of such traditional solutions is that the imaging of structured light 3D cameras is unstable, and the quality of 3D face images is easily affected by objective factors such as environmental factors and distance overrange, which in turn affects the stability of such 3D face recognition solutions. Figure 4 One or more embodiments of this specification provide a 3D face image with poor imaging effect in an application scenario. Figure 4 It can be seen that due to environmental factors, distance exceeding the range and other reasons, the quality of the 3D face image finally collected is poor, making it difficult to perform accurate face recognition.
[0054] Based on this, Figure 1 The process in may include the following steps:
[0055] S102: After the user's face is in a predetermined face placement position, a 3D face image is collected based on depth information of the user's face.
[0056] When users are performing business (e.g., face payment, face access control), the face recognition process is triggered, and the user's face image needs to be collected. Users can trigger face recognition on general devices such as smartphones and personal PCs, or on dedicated face recognition devices (e.g., face recognition vending machines, self-service face recognition ordering machines, face recognition access control devices). The face placement position usually refers to the preset area in front of the device camera. The position of the dedicated device in the space is relatively fixed, so the corresponding face placement position is also relatively fixed. On the device, in order to facilitate users to quickly find the face placement position, the face placement position can be highlighted for users in the form of a box, circle, etc.
[0057] A 3D face image can be a stereoscopic face image generated in a three-dimensional space, which can store and display the specific shape of the face image in the three-dimensional space to the user. However, in practical applications, if all 3D face images are in this form, a large amount of storage space and computing resources will be occupied. Therefore, a 2D image with depth information is usually used to collect and store 3D face images. At this time, it can not only meet the recognition and computing requirements of 3D face images, but also occupy less storage space.
[0058] Although the structured light 3D camera is used as an example for explanation in this article, in actual application, no matter what type of 3D camera, the face recognition method mentioned in the embodiments of this specification can be used, and it is not limited to the structured light 3D camera or the 3D camera with poor imaging effect. If the imaging effect of the 3D face image collected by the 3D camera is poor, the accuracy of face recognition can naturally be improved by the face recognition method mentioned in the embodiments of this specification. If the imaging effect of the 3D face image collected by the 3D camera is good, the accuracy and reliability of face recognition can still be further improved by the face recognition method mentioned in the embodiments of this specification.
[0059] S104: performing coordinate transformation on the 3D face image to obtain a 3D face point cloud corresponding to the user's face.
[0060] Since the 3D face image is collected based on depth information (for example, when the 3D face image is a 2D face depth map), it needs to be converted into a 3D face point cloud to facilitate subsequent processing of the 3D face image.
[0061] Specifically, Figure 5A schematic diagram of coordinate transformation in an application scenario is provided for one or more embodiments of the present specification. The 2D face depth map collected by a 3D camera usually carries a large amount of noise. At this time, the 2D face depth map is subjected to noise reduction processing, for example, noise reduction processing is performed by bilateral filtering. Among them, bilateral filtering (Bilateral filter) is a nonlinear filtering method, which is a compromise processing that combines the spatial proximity and pixel value similarity of the image, while considering the spatial domain information and grayscale similarity to achieve the purpose of edge preservation and noise reduction. According to the depth information corresponding to each pixel in the denoised 2D face depth map, coordinate transformation is performed to convert each pixel from a 2D pixel coordinate system to a 3D space coordinate system to obtain a 3D point cloud corresponding to the user.
[0062] In the process of collecting 3D face images, due to the user's standing position, the collected 3D face images may contain other parts of the user, such as Figure 5 The 3D face image shown in also includes the user's upper body. At this time, the converted 3D point cloud also includes the point cloud corresponding to the user's upper body. Based on this, Figure 6 A schematic diagram of target detection and key point detection in an application scenario provided for one or more embodiments of this specification, wherein a pre-trained 3D target detection model is used to perform target detection in a 3D point cloud and extract a 3D face point cloud corresponding to the user. The 3D target detection model can be obtained based on deep learning training, such as a convolutional neural network.
[0063] S106: According to the 3D face point cloud, the fitting parameters required for 3D face reconstruction are predicted using a pre-trained fitting parameter prediction model.
[0064] In traditional solutions, 3D face reconstruction can be performed in other ways. For example, a certain number of 3D faces are collected in advance to obtain the average shape and average texture of the faces, and then the eigenvectors of the covariance matrix arranged in descending order of eigenvalues are obtained to represent the face shape and texture information, so that a 3D model can be generated in advance. After obtaining the user's 2D face image, the rendering parameters are obtained based on it, and the rendering parameters are optimized together with the pre-generated 3D model, so that the final 3D image is as close as possible to the user's 2D face image, so as to achieve the purpose of roughly restoring 3D face information from the 2D face image. However, this solution is highly dependent on the quality of the 2D face image itself. When the quality of the 2D face image is poor, the restored 2D face information is naturally poor.
[0065] Figure 7A schematic diagram of the implementation process of 3D parametric face data in an application scenario is provided for one or more embodiments of this specification. In this solution, parameter prediction is no longer performed through a 2D face image, but a 3D face point cloud is used as input, and the fitting parameters required for 3D face reconstruction are predicted through a pre-trained fitting parameter prediction model. Compared with a 2D face image, the data contained in the 3D face point cloud is more comprehensive and more accurate, thereby increasing the accuracy of the predicted fitting parameters. Among them, the fitting parameter prediction can be obtained by training a neural network model using deep learning.
[0066] S108: According to the fitting parameters, a preset 3D reference face mesh is fitted to obtain 3D parameterized face data, wherein the 3D reference face mesh includes a plurality of vertices, the plurality of vertices are connected to each other, and each of the vertices is respectively provided with face semantics.
[0067] Figure 7 A schematic diagram of the implementation process of 3D parameterized face data in an application scenario provided for one or more embodiments of this specification, a 3D reference face mesh is pre-constructed, and the 3D reference face mesh is different from the 3D model in the traditional solution, in which multiple vertices are included, and multiple vertices are connected, and each vertex is respectively set with face semantics. At this time, it can be considered that the 3D reference face network has parameterized each vertex contained therein, so after obtaining the fitting parameters, fitting processing can be performed based on the fitting parameters to obtain 3D parameterized face data. For example, the fitting parameters can be Figure 7 α1, α2, ..., α shown in N After obtaining the fitting parameters, Perform fitting processing. Among them, S new Represents parameterized 3D face data, Represents the user's original 3D face point cloud, s i is the representation of face shape in different dimensions, α i For i The corresponding coefficient.
[0068] Parameterization in this scheme can be understood as setting corresponding parameterization rules for each vertex in the 3D reference face mesh (for example, the number of vertices, the connection relationship between vertices, the facial semantics of the vertices, the position coordinates of key points, the distance between edge points and other points, etc.), and taking the fitting parameters as input parameters. The fitting of the 3D reference face mesh does not change the set parameterization rules, so the output 3D parameterized face data still conforms to the parameterization rules, which is difficult to achieve with traditional schemes.
[0069] Specifically, some of the parameterized rules are used as examples for explanation. In the 3D reference face mesh, a preset number of vertices (for example, 15,000 vertices) are set, and each vertex is given a clear face semantics (for example, the 5000th vertex represents the nose tip of the face), and there is a fixed first connection topological relationship between the vertices, and the connection relationship between the vertices remains unchanged.
[0070] After obtaining the fitting parameters, the position coordinates of at least some of the vertices in the 3D reference face mesh are adjusted based on the basis vector weights therein while maintaining the first connection topological relationship. In this way, in the obtained 3D parameterized face data, the total number of vertices, the first connection topological relationship between the vertices, and the face semantics represented by each vertex do not change, and the parameterization of the 3D face image is achieved through the fitting parameters and the 3D reference face mesh. Figure 8 A schematic diagram of 3D parametric facial data in an application scenario is provided for one or more embodiments of the present specification. It can be clearly observed from the figure that the detail clarity of the 3D face is better than that of the original 3D face point cloud, thereby increasing the accuracy of face recognition.
[0071] S110: Identify the user according to the 3D parameterized face data.
[0072] The user takes a photo of his own face through the corresponding terminal in advance, and stores it in the 3D face database as 3D backup face data. Generally speaking, the face photo taken by the user through a 3D camera is parameterized in the manner described above, and the processing result is used as the 3D backup face data. However, when some users use terminals such as smartphones to keep a backup, it may be difficult to take 3D face photos due to camera reasons. At this time, the fitting parameters of the 2D face photos taken can be predicted to obtain 3D parameterized face data as 3D backup face data. Among them, the prediction of fitting parameters through 2D face photos can be implemented by training a neural network model similar to the above.
[0073] In one or more embodiments of the present specification, the 3D parameterized facial data is compared with each 3D retained facial data in a 3D face database, the 3D retained facial data with the highest similarity is selected, and the user is identified based on the user information corresponding to the 3D retained facial data.
[0074] Specifically, the 3D face database contains 3D retained face data corresponding to each user. When performing face recognition, the 3D parameterized face data is compared with each 3D retained face data to determine the similarity between the two. The 3D retained face data with the highest similarity is used as the user's recognition result, thereby verifying the user's identity information. Fig. 9 A schematic diagram of confirming facial similarity in an application scenario is provided for one or more embodiments of the present specification. In the 3D parameterized facial data and the 3D retained facial data, each vertex has a corresponding position coordinate (x, y, z). The Euclidean distance between each corresponding vertex (in the 3D parameterized facial data, each vertex has a corresponding facial semantics. Two vertices with the same facial semantics in the 3D parameterized facial data and the 3D retained facial data are considered to be corresponding vertices) is calculated. The smaller the Euclidean distance, the closer the positions of the corresponding vertices in the grid, and the higher the similarity. After obtaining the Euclidean distances between all corresponding vertices, the similarity between the two can be obtained by accumulation, averaging, etc. For example Fig. 9 As shown in , there are 3D facial data corresponding to 3 users (user 1, user 2 and user 3), and their corresponding Euclidean distances are 0.02, 0.4 and 0.3 respectively. Here, the 3D facial data with the smallest Euclidean distance of user 1 is selected as the 3D facial data with the highest similarity for face recognition. Of course, it has been explained above that target detection is performed in the user's 3D point cloud to obtain a 3D facial point cloud. Generally speaking, in the target detection process, the collected 3D facial point clouds have actually been aligned, so when comparing the similarity, the Euclidean distance between the corresponding vertices can be directly obtained to represent the similarity.
[0075] Furthermore, when determining the similarity, if the 3D facial data is stored earlier and the user's face shape has changed significantly due to some objective reasons, the Euclidean distance calculated by accumulation, averaging, etc. may no longer be accurate.
[0076] Based on this, the 3D parameterized face data (of course, also including the 3D retained face data) is pre-divided into multiple face areas, each area is set with fixed vertices, for example, 0-999 is the forehead area, 1000-1999 is the left eyebrow area... 14000-15000 is the edge area, etc., and the face change level is set for each face area. The face change level refers to the degree of change of each area over time under the influence of natural physiological factors (excluding human factors such as surgery and injury). The higher the face change level, the faster the change. Generally speaking, the edge area has the highest corresponding face change level because it has no bone support.
[0077] At this time, for each face region in the 3D parameterized face data, it is compared with the corresponding face region in the 3D retained face data to obtain the first sub-similarity for the face region. The first sub-similarity can be determined based on the Euclidean distance between vertices, similar to the similarity in the above text. At this time, based on the first sub-similarity, the influence of time on the face shape is considered, and the first sub-similarity is compensated according to the retention time of the 3D retained face data and the face change level of the face region to obtain the second sub-similarity. Among them, the longer the retention time and the higher the face change level, the greater the degree of change in the face shape may be, and the higher the degree of compensation is, and the degree of compensation is positively correlated with the retention time and the face change level. After obtaining the second sub-similarity of all face regions, the similarity between the 3D parameterized face data and the 3D retained face data can be obtained by summing, averaging, etc., so that the influence of time on the face shape can be combined to more accurately perform face recognition on users with longer retention time.
[0078] In one or more embodiments of the present specification, it has been described above that target detection is performed in the user's 3D point cloud, the user's 3D face point cloud is identified, and fitting parameters are predicted based on the 3D face point cloud. In order to increase the prediction accuracy of the fitting parameters and the smoothness of the subsequent fitting process of the 3D reference face mesh, after obtaining the user's 3D face point cloud, a pre-trained face key point detection model (for example, it can be similar to the 3D target detection model and generated by convolutional neural network training) can be used to perform key point detection on the 3D face point cloud to extract the key point information of the user's face. For example, Figure 6 As shown, key point information may include the tip of the nose, the corner of the mouth, the pupil, etc.
[0079] After extracting the key point information, the fitting parameters are predicted through the key point information and the 3D face point cloud, so as to improve the prediction accuracy of the fitting parameters. Figure 7 As shown, when fitting the 3D reference face mesh, the key point information may be fitted first, and in the fitting process of other vertices, adaptive smoothing may be performed based on the key point information, thereby increasing the smoothness of the 3D parameterized face data.
[0080] Furthermore, although keeping 3D facial data as a backup can improve the privacy and security of users, it is still difficult to meet the requirements for some scenarios with higher security levels, and the storage space and computing resources used are also large.
[0081] Based on this, after obtaining the corresponding key point information in the 3D parameterized face data and the 3D retained face data, a second connection topology relationship is generated according to the position coordinates of each key point, and the second connection topology relationship only contains the key point information and no longer contains the information of other vertices. The key point information is the vertex of the core part of the face, which can reflect the basic information of the face. When the similarity between the 3D parameterized face data and each 3D retained face data is determined by the second connection topology relationship (for example, the similarity is determined by the Euclidean distance between the two, the calculation speed when the two are compared when fitting, the consumption of computing resources, etc.), a more accurate recognition effect can be achieved under the premise of consuming less computing resources, thereby selecting the specified 3D retained face data whose similarity exceeds the preset threshold to perform 3D face recognition on the user.
[0082] At this time, determine the security level corresponding to the specified 3D retained face data, which can be pre-set based on user needs, business needs, etc. When the security level is low (lower than the preset level), it can be considered that the specified 3D retained face data obtained by screening through the second connection topology relationship is a preliminary screening process. Here, a more detailed first connection topology relationship is used to perform a secondary screening on the specified 3D retained face data to select the specified 3D retained face data with the highest similarity, which is used for 3D face recognition of the user, so as to perform a large-scale preliminary screening through the second connection topology relationship with less computing resource consumption, thereby reducing the overall computing resource consumption. When the security level is high (higher than the preset level), in the 3D face database, the first connection topology relationship in the specified 3D retained face is deleted, and the second connection topology relationship is used as a replacement and stored in the 3D retained face data. Of course, in the 3D retained face data, other vertices except key points can be retained or deleted based on user needs. At this time, face recognition is performed only based on the second connection topology relationship, which can further increase the privacy and security of the user.
[0083] Fig.10 Provided for one or more embodiments of this specification, a flow chart of a face recognition method in an application scenario. When this solution is applied to offline face payment, first in the 3D face data acquisition and preprocessing module, the user's 2D face depth map is collected by a structured light 3D camera, and preprocessed (for example, noise reduction, coordinate conversion), and 3D face information (including 3D face point cloud and key point information) is obtained through face target recognition. Next, in the 3D face parameterization module, 3D face parameterization is performed through a preset 3D reference face grid to obtain 3D parameterized face data. Finally, in the 3D face database in the parameterized 3D face recognition module, each 3D retained face data is determined, and the user information is finally determined by comparative retrieval.
[0084] Based on the same idea, one or more embodiments of this specification also provide devices and apparatuses corresponding to the above methods, such as Fig.11 , Fig.12 shown.
[0085] Fig.11 A schematic diagram of the structure of a face recognition device provided for one or more embodiments of this specification, the device comprising:
[0086] The acquisition module 1102 acquires a 3D face image based on the depth information of the user's face after the user's face is in a predetermined face placement position;
[0087] A coordinate conversion module 1104 performs coordinate conversion on the 3D face image to obtain a 3D face point cloud corresponding to the user's face;
[0088] A fitting parameter prediction module 1106 predicts the fitting parameters required for 3D face reconstruction based on the 3D face point cloud using a pre-trained fitting parameter prediction model;
[0089] The parameterization module 1108 performs fitting processing on a preset 3D reference face mesh according to the fitting parameters to obtain 3D parameterized face data, wherein the 3D reference face mesh includes a plurality of vertices, the plurality of vertices are connected to each other, and each of the vertices is respectively provided with face semantics;
[0090] The recognition module 1110 recognizes the user according to the 3D parameterized face data.
[0091] Optionally, the number of vertices in the 3D reference face mesh is fixed, and the connection relationship between the multiple vertices is a fixed first connection topology relationship;
[0092] The parameterization module 1108 adjusts the position coordinates of at least some vertices in the 3D reference face mesh according to the basis vector weights in the fitting parameters while maintaining the first connection topological relationship to obtain 3D parameterized face data.
[0093] Optionally, the 3D face image is a 2D face depth image;
[0094] The coordinate conversion module 1104 performs noise reduction processing on the 2D face depth map;
[0095] Performing coordinate transformation according to the depth information corresponding to each pixel in the denoised 2D face depth map, so as to transform each pixel from a 2D pixel coordinate system into a 3D space coordinate system, and obtaining a 3D point cloud corresponding to the user;
[0096] The 3D point cloud is subjected to target detection by using a pre-trained 3D target detection model, so as to determine the 3D face point cloud corresponding to the user.
[0097] Optionally, it also includes:
[0098] A key point recognition module 1112 performs key point detection on the 3D face point cloud using a pre-trained face key point detection model to determine key point information of the user's face;
[0099] The fitting parameter prediction module 1106 predicts the fitting parameters required for 3D face reconstruction based on the 3D face point cloud and the key point information using a pre-trained fitting parameter prediction model.
[0100] Optionally, the recognition module 1110 compares the 3D parameterized face data with each 3D retained face data in the 3D face database to determine the similarity between the 3D parameterized face data and each 3D retained face data, wherein the similarity is determined according to the Euclidean distance between each corresponding vertex in the 3D parameterized face data and the 3D retained face data;
[0101] The user is subjected to 3D face recognition based on the 3D retained face data with the highest similarity.
[0102] Optionally, the recognition module 1110 determines a plurality of divided face regions in the 3D parametric face data, and determines a face change level corresponding to each of the face regions;
[0103] For each of the face regions in the 3D parameterized face data, the face region is compared with the face region corresponding to each 3D retained face data in the 3D face database to obtain a first sub-similarity corresponding to the face region, and the first sub-similarity is compensated according to the retention time of each 3D retained face data and the corresponding face change level to obtain a second sub-similarity, wherein the degree of compensation is positively correlated with the retention time and the face change level;
[0104] According to the second sub-similarity of each facial region, the similarity between the 3D parameterized facial data and each of the 3D retained facial data is obtained.
[0105] Optionally, the recognition module 1110 determines key point information corresponding to the 3D parameterized face data and each 3D retained face data in the 3D face database;
[0106] Generate corresponding second connection topological relationships according to the key point information;
[0107] Determining the similarity between the 3D parameterized face data and each of the 3D retained face data through the second connection topological relationship;
[0108] In the 3D face database, the designated 3D retained face data whose similarity exceeds a preset threshold is selected, so as to perform 3D face recognition on the user through the designated 3D retained face data.
[0109] Optionally, the connection relationship between the multiple vertices in the 3D reference face mesh is a fixed first connection topology relationship;
[0110] The recognition module 1110 determines the security level corresponding to the specified 3D retained face data;
[0111] If the security level is lower than a preset level, selecting the designated 3D retained face data with the highest similarity from the designated 3D retained face data according to the first connection topology relationship, for use in performing 3D face recognition on the user;
[0112] If the security level is higher than the preset level, in the 3D face database, the first connection topology relationship in the specified 3D retained face is deleted, and the second connection topology relationship is used as a replacement and stored in the specified 3D retained face data.
[0113] Fig.12 A schematic diagram of the structure of a face recognition device provided for one or more embodiments of this specification, the device comprising:
[0114] at least one processor; and,
[0115] a memory communicatively connected to the at least one processor; wherein,
[0116] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:
[0117] After the user's face is in a predetermined face placement position, collecting a 3D face image based on the depth information of the user's face;
[0118] Performing coordinate transformation on the 3D face image to obtain a 3D face point cloud corresponding to the user's face;
[0119] According to the 3D face point cloud, the fitting parameters required for 3D face reconstruction are predicted by a pre-trained fitting parameter prediction model;
[0120] According to the fitting parameters, a preset 3D reference face mesh is fitted to obtain 3D parameterized face data, wherein the 3D reference face mesh includes a plurality of vertices, the plurality of vertices are connected to each other, and each of the vertices is respectively provided with face semantics;
[0121] The user is identified according to the 3D parameterized face data.
[0122] Based on the same idea, one or more embodiments of this specification also provide a non-volatile computer storage medium corresponding to the above method, storing computer executable instructions, wherein the computer executable instructions are configured as follows:
[0123] After the user's face is in a predetermined face placement position, collecting a 3D face image based on the depth information of the user's face;
[0124] Performing coordinate transformation on the 3D face image to obtain a 3D face point cloud corresponding to the user's face;
[0125] According to the 3D face point cloud, the fitting parameters required for 3D face reconstruction are predicted by a pre-trained fitting parameter prediction model;
[0126] According to the fitting parameters, a preset 3D reference face mesh is fitted to obtain 3D parameterized face data, wherein the 3D reference face mesh includes a plurality of vertices, the plurality of vertices are connected to each other, and each of the vertices is respectively provided with face semantics;
[0127] The user is identified according to the 3D parameterized face data.
[0128] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.
[0129] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.
[0130] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0131] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0132] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0133] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0136] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0137] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0138] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0139] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0140] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.
[0141] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0142] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0143] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.
Claims
1. A face recognition method, comprising: After the user's face is in a predetermined face placement position, collecting a 3D face image based on the depth information of the user's face; Performing coordinate transformation on the 3D face image to obtain a 3D face point cloud corresponding to the user's face; According to the 3D face point cloud, the fitting parameters required for 3D face reconstruction are predicted by a pre-trained fitting parameter prediction model; According to the fitting parameters, a preset 3D reference face mesh is fitted to obtain 3D parameterized face data, wherein the 3D reference face mesh includes a plurality of vertices, the plurality of vertices are connected to each other, and each of the vertices is respectively provided with face semantics; The user is identified according to the 3D parameterized face data, specifically comprising: determining a plurality of face regions that have been divided in the 3D parameterized face data, and determining a face change level corresponding to each of the face regions; for each of the face regions in the 3D parameterized face data, comparing the face region with a face region corresponding to each 3D retained face data in a 3D face database to obtain a first sub-similarity corresponding to the face region, and compensating the first sub-similarity according to a retention time of each 3D retained face data and a corresponding face change level to obtain a second sub-similarity, wherein the degree of compensation is positively correlated with the retention time and the face change level; obtaining a similarity between the 3D parameterized face data and each of the 3D retained face data according to the second sub-similarity of each face region; and performing 3D face recognition on the user according to the 3D retained face data with the highest similarity.
2. The method according to claim 1, wherein the number of vertices in the 3D reference face mesh is fixed, and the connection relationship between the plurality of vertices is a fixed first connection topology relationship; The step of performing fitting processing on a preset 3D reference face mesh according to the fitting parameters to obtain 3D parameterized face data specifically includes: According to the basis vector weights in the fitting parameters, on the basis of maintaining the first connection topological relationship, the position coordinates of at least some vertices in the 3D reference face mesh are adjusted to obtain 3D parameterized face data.
3. The method according to claim 1, wherein the 3D face image is a 2D face depth image; The coordinate transformation of the 3D face image to obtain a 3D face point cloud corresponding to the user's face specifically includes: Performing noise reduction processing on the 2D face depth map; Performing coordinate transformation according to the depth information corresponding to each pixel in the denoised 2D face depth map, so as to transform each pixel from a 2D pixel coordinate system into a 3D space coordinate system, and obtaining a 3D point cloud corresponding to the user; The 3D point cloud is subjected to target detection by using a pre-trained 3D target detection model, so as to determine the 3D face point cloud corresponding to the user.
4. The method of claim 3, wherein after performing target detection on the 3D point cloud using a pre-trained 3D target detection model to determine a 3D face point cloud corresponding to the user, the method further comprises: Performing key point detection on the 3D face point cloud using a pre-trained face key point detection model to determine key point information of the user's face; The step of predicting the fitting parameters required for 3D face reconstruction based on the 3D face point cloud by using a pre-trained fitting parameter prediction model specifically includes: According to the 3D face point cloud and the key point information, the fitting parameters required for 3D face reconstruction are predicted by a pre-trained fitting parameter prediction model.
5. The method as claimed in claim 1, wherein the similarity is determined based on the Euclidean distance between corresponding vertices in the 3D parameterized face data and the 3D retained face data.
6. The method according to claim 4, wherein the identifying the user according to the 3D parameterized face data comprises: Determine key point information corresponding to the 3D parametric face data and each 3D retained face data in the 3D face database; Generate corresponding second connection topological relationships according to the key point information; Determining the similarity between the 3D parameterized face data and each of the 3D retained face data through the second connection topological relationship; In the 3D face database, the designated 3D retained face data whose similarity exceeds a preset threshold is selected, so as to perform 3D face recognition on the user through the designated 3D retained face data.
7. The method according to claim 6, wherein the connection relationship between the plurality of vertices in the 3D reference face mesh is a fixed first connection topology relationship; The performing 3D face recognition on the user by using the specified 3D retained face data specifically includes: Determine the security level corresponding to the specified 3D retained facial data; If the security level is lower than a preset level, selecting the designated 3D retained face data with the highest similarity from the designated 3D retained face data according to the first connection topology relationship, for use in performing 3D face recognition on the user; If the security level is higher than the preset level, in the 3D face database, the first connection topology relationship in the specified 3D retained face is deleted, and the second connection topology relationship is used as a replacement and stored in the specified 3D retained face data.
8. A face recognition device, comprising: A collection module, after the user's face is in a predetermined face placement position, collects a 3D face image based on the depth information of the user's face; A coordinate conversion module performs coordinate conversion on the 3D face image to obtain a 3D face point cloud corresponding to the user's face; A fitting parameter prediction module, which predicts the fitting parameters required for 3D face reconstruction based on the 3D face point cloud using a pre-trained fitting parameter prediction model; A parameterization module, which performs fitting processing on a preset 3D reference face mesh according to the fitting parameters to obtain 3D parameterized face data, wherein the 3D reference face mesh includes a plurality of vertices, the plurality of vertices are connected to each other, and each of the vertices is respectively provided with face semantics; The recognition module recognizes the user according to the 3D parameterized face data, specifically comprising: determining a plurality of face regions that have been divided in the 3D parameterized face data, and determining a face change level corresponding to each of the face regions; for each of the face regions in the 3D parameterized face data, comparing the face region with a face region corresponding to each 3D retained face data in a 3D face database to obtain a first sub-similarity corresponding to the face region, and compensating the first sub-similarity according to a retention time of each 3D retained face data and a corresponding face change level to obtain a second sub-similarity, wherein the degree of compensation is positively correlated with the retention time and the face change level; obtaining a similarity between the 3D parameterized face data and each of the 3D retained face data according to the second sub-similarity of each face region; and performing 3D face recognition on the user according to the 3D retained face data with the highest similarity.
9. The device according to claim 8, wherein the number of vertices in the 3D reference face mesh is fixed, and the connection relationship between the plurality of vertices is a fixed first connection topology relationship; The parameterization module adjusts the position coordinates of at least some vertices in the 3D reference face mesh according to the basis vector weights in the fitting parameters while maintaining the first connection topological relationship to obtain 3D parameterized face data.
10. The device according to claim 8, wherein the 3D face image is a 2D face depth image; The coordinate conversion module performs noise reduction processing on the 2D face depth map; Performing coordinate transformation according to the depth information corresponding to each pixel in the denoised 2D face depth map, so as to transform each pixel from a 2D pixel coordinate system into a 3D space coordinate system, and obtaining a 3D point cloud corresponding to the user; The 3D point cloud is subjected to target detection by using a pre-trained 3D target detection model, so as to determine the 3D face point cloud corresponding to the user.
11. The apparatus of claim 10, further comprising: A key point recognition module performs key point detection on the 3D face point cloud using a pre-trained face key point detection model to determine key point information of the user's face; The fitting parameter prediction module predicts the fitting parameters required for 3D face reconstruction based on the 3D face point cloud and the key point information through a pre-trained fitting parameter prediction model.
12. The device as described in claim 8, wherein the similarity is determined based on the Euclidean distance between corresponding vertices in the 3D parameterized face data and the 3D retained face data.
13. The device according to claim 11, wherein the recognition module determines key point information corresponding to the 3D parameterized face data and each 3D retained face data in the 3D face database; Generate corresponding second connection topological relationships according to the key point information; Determining the similarity between the 3D parameterized face data and each of the 3D retained face data through the second connection topological relationship; In the 3D face database, the designated 3D retained face data whose similarity exceeds a preset threshold is selected, so as to perform 3D face recognition on the user through the designated 3D retained face data.
14. The apparatus according to claim 13, wherein the connection relationship between the plurality of vertices in the 3D reference face mesh is a fixed first connection topology relationship; The recognition module determines the security level corresponding to the specified 3D retained face data; If the security level is lower than a preset level, selecting the designated 3D retained face data with the highest similarity from the designated 3D retained face data according to the first connection topology relationship, for use in performing 3D face recognition on the user; If the security level is higher than the preset level, in the 3D face database, the first connection topology relationship in the specified 3D retained face is deleted, and the second connection topology relationship is used as a replacement and stored in the specified 3D retained face data.
15. A face recognition device, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: After the user's face is in a predetermined face placement position, collecting a 3D face image based on the depth information of the user's face; Performing coordinate transformation on the 3D face image to obtain a 3D face point cloud corresponding to the user's face; According to the 3D face point cloud, the fitting parameters required for 3D face reconstruction are predicted by a pre-trained fitting parameter prediction model; According to the fitting parameters, a preset 3D reference face mesh is fitted to obtain 3D parameterized face data, wherein the 3D reference face mesh includes a plurality of vertices, the plurality of vertices are connected to each other, and each of the vertices is respectively provided with face semantics; The user is identified according to the 3D parameterized face data, specifically comprising: determining a plurality of face regions that have been divided in the 3D parameterized face data, and determining a face change level corresponding to each of the face regions; for each of the face regions in the 3D parameterized face data, comparing the face region with a face region corresponding to each 3D retained face data in a 3D face database to obtain a first sub-similarity corresponding to the face region, and compensating the first sub-similarity according to a retention time of each 3D retained face data and a corresponding face change level to obtain a second sub-similarity, wherein the degree of compensation is positively correlated with the retention time and the face change level; obtaining a similarity between the 3D parameterized face data and each of the 3D retained face data according to the second sub-similarity of each face region; and performing 3D face recognition on the user according to the 3D retained face data with the highest similarity.
