3D Face Recognition Method for Chip, Face Recognition Chip, Readable Storage Medium

Abstract

The present invention discloses a 3D face recognition method for a chip, a face recognition chip, and a readable storage medium, including obtaining a face region RGB depth image of a target face, where the RGB depth image includes RGB image information and depth information of the face region, fusing the depth information and the RGB image information of the face region, and identifying whether the target face is a 3D face according to the fused information. When the target face is a 3D face, inputting the RGB image information of the face region into a 2D face recognition model for face recognition. When the target face is not a 3D face, returning to the step of obtaining the face region RGB depth image of the target face, thereby improving the security of face recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and particularly relates to a 3D (3 Dimensions) face recognition method for a chip, a face recognition chip, and a readable storage medium. Background Art

[0002] Face recognition technology is based on the facial features of a person, extracts the identity features in each face, and compares them with known faces to identify the identity of each face. Face recognition technology is widely used in fields such as unlocking and payment.

[0003] Currently, 2D (2 Dimensions) face recognition technology is adopted in face recognition technology. However, since 2D face recognition can only obtain the planar information of the face, it can be cracked by pictures, videos, etc., and there are relatively large security risks. Summary of the Invention

[0004] Based on this, the present invention provides a 3D face recognition method for a chip, a face recognition chip, and a readable storage medium, which can improve the security of face recognition.

[0005] In a first aspect, a 3D face recognition method for a chip is provided, including:

[0006] Obtain a face region RGB depth image of a target face, where the RGB depth image includes RGB image information and depth information of the face region;

[0007] Fuse the depth information and RGB image information of the face region, and identify whether the target face is a 3D face according to the fused information; wherein, the RGB image information and depth information are fused by using the two-in-one network structure, and the two-in-one network structure includes two encryption networks, one decryption network, and one recognition network. Both encryption networks are CNN convolutional neural networks and the weights are shared. One encryption network is used to input and learn the RGB image information, and the other encryption network is used to input and learn the depth information; the decryption network is used to perform a deconvolution operation on the output information of the two encryption networks to obtain output features, so as to realize the fusion of the RGB image information and the depth information, and the recognition network recognizes the output features to obtain the 3D face recognition output result;

[0008] When the target face is a 3D face, input the face region RGB image information into a 2D face recognition model for face recognition. When the target face is not a 3D face, return to the step of obtaining the face region RGB depth image of the target face.

[0009] In one embodiment, when the output result of the 3D face recognition is 0, the target face is a 3D face, and when the output result is 1, the target face is a non-3D face.

[0010] In one embodiment, the step of obtaining the RGB depth image of the face area of the target face includes:

[0011] Obtain an RGB face area image, obtain the two-dimensional coordinates and three-dimensional coordinates of the face feature points in the RGB face area image. The face feature points include the feature points on the edge of the face contour, and also include the feature points of the eyebrow contour, the feature points of the nose contour, the feature points of the eye contour and / or the feature points of the mouth contour;

[0012] Perform curve fitting on the feature points on the edge of the face contour to obtain a complete RGB image of the face area, and obtain the corresponding depth information according to the complete RGB image of the face area. The RGB depth image of the face area is obtained according to the complete RGB image of the face area and the corresponding depth information.

[0013] In one embodiment, the formula for performing curve fitting on the feature points on the edge of the face contour is:

[0014]

[0015] A is a 2×4 affine transformation matrix of each feature point on the edge of the face contour, X1, Y1, Z1 are the three-dimensional coordinates of the first set of feature points on the edge of the face contour, is the two-dimensional coordinates of the second set of feature points on the edge of the face contour. Among them, the first set of feature points is mapped into the two-dimensional space, and the feature point closest to it is used as the second set of feature points to minimize the error D.

[0016] In one embodiment, the structure of the two-dimensional face recognition model sequentially includes an input layer, a Conv convolutional layer, a Relu activation function layer, a pooling layer, a fully connected layer, and a normalization layer along the data transmission direction. Among them, there are two or more Conv convolutional layers and Relu activation function layers, and the numbers are the same. The number of pooling layers is one less than that of the Conv convolutional layer and the Relu activation function layer. The number of fully connected layers is one. Along the data transmission direction, the first Conv convolutional layer is used to access the data output by the input layer. The data output by the second Relu activation function layer is transmitted to the first pooling layer, and then to the third Conv convolutional layer. The last pooling layer is used to output the data to the fully connected layer. The fully connected layer is used to map the data into a 256-dimensional feature space and output it to the normalization layer. The normalization layer is used to normalize the face features into a range with a radius of 1.

[0017] In one embodiment, the convolution kernel size of each Conv convolutional layer is 3×3, and the convolution stride is 1.

[0018] In one embodiment, the following loss function is used during the training of the two-dimensional face recognition model:

[0019] Floss = κ 1 F1 + κ 2 F2

[0020]

[0021] where κ 1 and κ 2 are the weight factors of the loss functions F1 and F2 respectively, L1 represents the maximum distance between the training face features of each face category and the center point feature, λ is a balance factor in the range of 0.01 - 0.1, xi represents the feature of the i-th category of training face, pi represents the center point of the feature of the i-th category of training face, W1 is the connection weight vector of the i-th category of neurons, θ is the connection offset of the i-th category of neurons, W2 is the connection weight vector of the j-th category of neurons, is the connection offset of the j-th category of neurons.

[0022] In one embodiment, after the step of obtaining the RGB depth image of the face region of the target face and before the step of inputting the RGB image information of the face region into the two-dimensional face recognition model for face recognition, the RGB depth image of the face region is smoothed.

[0023] In a second aspect, a face recognition chip is proposed, which includes a storage unit and a processing unit. A computer program is stored in the storage unit. When the computer program is executed by the processing unit, the processing unit executes the steps of the method described in any of the above embodiments.

[0024] In a third aspect, one or more non-volatile readable storage media storing computer-readable instructions are proposed, characterized in that when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method described in any of the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1Schematic flow chart of the 3D face recognition method of the chip in an embodiment of the present application;

[0027] Figure 2 Schematic structural diagram of the two-in-one network structure in an embodiment of the present application;

[0028] Figure 3 Schematic structural diagram of the 2D face recognition model in an embodiment of the present application;

[0029] Figure 4 Schematic structural diagram of the face recognition chip in an embodiment of the present application. Detailed implementation manners

[0030] As described in the background art, currently, 2D (2 Dimensions) face recognition technology is adopted in face recognition technology. However, since 2D face recognition can only obtain the planar information of the human face and can be cracked by pictures, videos, etc., there are relatively large security risks.

[0031] The embodiments of the present application propose a 3D face recognition method for a chip, a face recognition chip, and a readable storage medium, which can improve the security of face recognition.

[0032] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all the implementation manners consistent with the present application. On the contrary, they are only examples of the devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0033] It should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or device. Without further limitations, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including that element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined according to their explanations in the specific embodiments or further in combination with the context of the specific embodiments.

[0034] It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this document, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining". Furthermore, as used in this document, the singular forms "a", "an", and "the" are also intended to include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising", "including" indicate the presence of the stated features, steps, operations, elements, components, items, types, and / or groups, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, types, and / or groups. The terms "or" and "and / or" as used herein are interpreted inclusively, or meaning any one or any combination. Thus, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C". An exception to this definition only occurs when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0035] It should be understood that although the steps in the flowcharts in the embodiments of this application are shown sequentially according to the indication of the arrows, these steps are not necessarily executed sequentially according to the order indicated by the arrows. Unless there is a clear indication in this document, the execution of these steps has no strict order restriction, and they can be executed in other orders. Moreover, at least a part of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments, and their execution order is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0036] It should be noted that in this document, step codes such as 102, 104, 106, etc. are used. The purpose is to more clearly and briefly express the corresponding content and do not constitute a substantial limitation in terms of order. Those skilled in the art may execute 104 first and then 102, etc. during specific implementation, but these should all be within the protection scope of this application.

[0037] Figure 1 is a schematic flowchart of a 3D face recognition method of a chip according to an embodiment of this application, as Figure 1 shown, the decryption method includes steps 102 to 106:

[0038] Step 102: Obtain the RGB-depth image of the face region of the target face, where the RGB-depth image contains the RGB image information and depth information of the face region.

[0039] Step 104: Fuse the depth information and RGB image information of the face region, and identify whether the target face is a 3D face based on the fused information.

[0040] In one embodiment, the two-in-one network structure is used to fuse the RGB image information and depth information. As Figure 2 shown, the two-in-one network structure includes two encryption networks, one decryption network, and one recognition network. Both encryption networks are CNN convolutional neural networks with shared weights. One encryption network is used to input and learn the RGB image information, and the other encryption network is used to input and learn the depth information. The decryption network is used to perform deconvolution operations on the output information of the two encryption networks to obtain output features, thereby realizing the fusion of the RGB image information and depth information. The recognition network identifies the output features to obtain the 3D face recognition output result, thereby realizing attacks on forged faces and non-faces.

[0041] Specifically, when the 3D face recognition output result is 0, the target face is a 3D face; when the output result is 1, the target face is a non-3D face.

[0042] Step 106: When the target face is a 3D face, input the RGB image information of the face region into the 2D face recognition model for face recognition. When the target face is a non-3D face, return to the step of obtaining the RGB-depth image of the face region of the target face.

[0043] In the embodiment of the present application, by obtaining and fusing the depth information and RGB image information of the face region, and identifying whether the target face is a 3D face based on the fused information, attacks on forged faces and non-faces are realized. When the target face is a 3D face, the RGB image information of the face region is input into the 2D face recognition model for face recognition, improving the security of face recognition.

[0044] Specifically, after the step of obtaining the RGB-depth image of the face region of the target face and before the step of inputting the RGB image information of the face region into the 2D face recognition model for face recognition, the RGB-depth image of the face region is smoothed to improve the recognition accuracy.

[0045] For step 102, in some embodiments, the step of obtaining the RGB-depth image of the face region of the target face includes: obtaining an RGB face region image, obtaining the two-dimensional coordinates and three-dimensional coordinates of the facial feature points in the RGB face region image, where the facial feature points include the feature points on the edge of the face contour, and also include the feature points of the eyebrow contour, the nose contour, the eye contour, and / or the mouth contour feature points, etc., and are not limited thereto. In these embodiments, when the facial feature points further include the feature points of the eyebrow contour, the nose contour, the eye contour, and / or the mouth contour feature points, it indicates that the obtained is the face region rather than the non-face region, which is beneficial to improving the accuracy of face recognition.

[0046] After determining that the two-dimensional coordinates and three-dimensional coordinates of the facial feature points are obtained from the RGB face region rather than the non-face region image, curve fitting is performed on the feature points on the edge of the face contour of the RGB face region, so as to obtain a complete RGB image of the face region, and the corresponding depth information is obtained according to the complete RGB image of the face region, and the RGB-depth image of the face region is obtained according to the complete RGB image of the face region and the corresponding depth information.

[0047] Specifically, obtaining the corresponding depth information according to the complete RGB image of the face region may be to register the depth image collected by the depth camera with the RGB image, and then obtain the depth information of the registered depth image. The depth camera may be set independently of the RGB camera or integrated with the RGB camera. For example, an RGB-D camera may be used, which can obtain both RGB images and depth images.

[0048] Specifically, the corresponding depth information may be obtained according to the mapping relationship between the curve obtained by fitting and the three-dimensional coordinates obtained above, so as to obtain the RGB-depth image information.

[0049] In one embodiment, the formula for performing curve fitting on the feature points on the edge of the face contour is:

[0050]

[0051] A is the 2×4 affine transformation matrix of each feature point on the edge of the face contour, and X1, Y1, Z1 are the three-dimensional coordinates of the first set of feature points on the edge of the face contour. is the two-dimensional coordinate of the second set of feature points on the edge of the face contour. Among them, the first set of feature points is mapped into the two-dimensional space, and the feature point closest to it is used as the second set of feature points to minimize the error D.

[0052] In this embodiment, the facial contour includes facial feature points, enabling a relatively accurate facial region to be obtained. Further, when the error D is minimized, the effect of curve fitting on the feature points of the facial contour edge is the best, further improving the accuracy of the obtained facial region.

[0053] In some embodiments, as Figure 3 shown, the structure of the two-dimensional face recognition model sequentially includes an input layer, a Conv convolutional layer, a Relu activation function layer, a pooling layer, a fully connected layer, and a normalization layer along the data transmission direction. Among them, there are more than two Conv convolutional layers and Relu activation function layers, and the numbers are the same. The number of pooling layers is one less than that of the Conv convolutional layer and the Relu activation function layer. The number of fully connected layers is one. Along the data transmission direction, the first Conv convolutional layer is used to access the data output by the input layer. The data output by the second Relu activation function layer is transmitted to the first pooling layer, and then to the third Conv convolutional layer. The last pooling layer is used to output the data to the fully connected layer. The fully connected layer is used to map the data to a 256-dimensional feature space and output it to the normalization layer. The normalization layer is used to normalize the face features to a range with a radius of 1. The mapped space is not limited to 256 dimensions and can also be 128 dimensions. Specifically, the convolution kernel size of each Conv convolutional layer is 3×3, and the convolution stride is 1. However, it is not limited to this.

[0054] In some embodiments, the following loss function is used during the training of the two-dimensional face recognition model:

[0055] Floss = κ 1 F1 + κ 2 F2

[0056]

[0057] Among them, κ 1 and κ 2 are the weight factors of the loss functions F1 and F2 respectively. L1 represents the maximum distance between the training face features of each face category and the center point feature. λ is a balance factor, which is in the range of 0.01 - 0.1. xi represents the feature of the i-th category of training face, pi represents the center point of the feature of the i-th category of training face, W1 is the connection weight vector of the i-th category of neurons, θ is the connection offset of the i-th category of neurons, W2 is the connection weight vector of the j-th category of neurons, is the connection offset of the j-th category of neurons.

[0058] These embodiments consider multiple losses, which is more comprehensive and makes the trained model more accurate. When this model is applied to face recognition, the accuracy will also be higher. Among them, the weight factors of the loss functions F1 and F2 can be dynamically adjusted according to the training speed to avoid too long training time.

[0059] An embodiment of the present application further provides a face recognition chip, as Figure 4 shown. The face recognition chip 400 includes a storage unit 410 and a processing unit 420. A computer program is stored in the storage unit 410. When the computer program is executed by the processing unit 420, the processing unit 420 is caused to execute the steps of the method described in any of the foregoing embodiments.

[0060] An embodiment of the present application further provides an electronic device including the face recognition chip of the embodiment of the present application. The electronic device may be any terminal device including a mobile phone, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), an in-vehicle computer, a wearable device, and the like.

[0061] One or more non-volatile readable storage media storing computer-readable instructions are also provided. The computer-readable instructions, when executed by one or more processors, cause the one or more processors to execute the steps of the method described in any of the foregoing embodiments.

[0062] A computer program product containing instructions, when running on a computer, causes the computer to execute the method described in any of the foregoing embodiments.

[0063] Any reference in the present application to a memory, storage, database, or other medium may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM).

[0064] Although the present application has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art based on a reading and understanding of this specification and the drawings. The present application includes all such modifications and variations and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the above-described components, the terms used to describe such components are intended to correspond to any component (unless otherwise indicated) that performs the specified function of the component (i.e., that is functionally equivalent), even if not structurally equivalent to the disclosed structure that performs the function in the exemplary implementations of the present specification shown herein. Moreover, although a particular feature of the present specification has been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of other implementations as may be desired and advantageous for a given or particular application. Also, insofar as the terms "comprising," "having," "containing," or variations thereof are used in the detailed description or claims, such terms are intended to include in a manner similar to the term "including." Further, it should be understood that the "plurality" referred to herein means two or more. For the steps mentioned herein, the numerical suffixes are only for the purpose of clearly expressing the embodiments and facilitating understanding, and do not completely represent the order of execution of the steps. The order should be set according to the logical relationship.

[0065] The above are only embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, such as the combination of technical features between embodiments, or direct or indirect application in other related technical fields, shall be included in the patent protection scope of the present application by the same token.

Claims

1. A 3D face recognition method for a chip, characterized in that, it includes: Obtain the RGB depth image of the face area of the target face, where the RGB depth image contains the RGB image information and depth information of the face area; Fuse the depth information and RGB image information of the face area, and identify whether the target face is a 3D face according to the fused information; among them, a two-in-one network structure is used for fusing RGB image information and depth information. The two-in-one network structure includes two encryption networks, one decryption network and one recognition network. Both encryption networks are CNN convolutional neural networks and the weights are shared. One encryption network is used to input and learn RGB image information, and the other encryption network is used to input and learn depth information; the decryption network is used to perform deconvolution operations on the output information of the two encryption networks to obtain output features, so as to realize the fusion of RGB image information and depth information. The recognition network identifies the output features to obtain the 3D face recognition output result; When the target face is a 3D face, input the RGB image information of the face area into a two-dimensional face recognition model for face recognition. When the target face is not a 3D face, return to the step of obtaining the RGB depth image of the face area of the target face; Among them, the step of obtaining the RGB depth image of the face area of the target face includes: Obtain the RGB face area image, and obtain the two-dimensional coordinates and three-dimensional coordinates of the face feature points in the RGB face area image. The face feature points include the feature points on the edge of the face contour, and also include the feature points of the eyebrow contour, the feature points of the nose contour, the eye contour feature and / or the feature points of the mouth contour; Perform curve fitting on the feature points on the edge of the face contour to obtain a complete RGB image of the face area, and obtain the corresponding depth information according to the complete RGB image of the face area. Obtain the RGB depth image of the face area according to the complete RGB image of the face area and the corresponding depth information; The formula for performing curve fitting on the feature points on the edge of the face contour is: Among them, A is a 2×4 affine transformation matrix of each feature point on the edge of the face contour, and X1, Y1, Z1 are the three-dimensional coordinates of the first feature point set on the edge of the face contour. are the two-dimensional coordinates of the second feature point set on the edge of the face contour. Among them, the first feature point set is mapped into the two-dimensional space, and the nearest feature point is used as the second feature point set to minimize the error D.

2. The method according to claim 1, characterized in that, When the 3D face recognition output result is 0, the target face is a 3D face. When the output result is 1, the target face is not a 3D face.

3. The method according to claim 1, characterized in that, The structure of the two-dimensional face recognition model sequentially includes an input layer, a Conv convolutional layer, a Relu activation function layer, a pooling layer, a fully connected layer, and a normalization layer along the data transmission direction. Among them, there are more than two Conv convolutional layers and Relu activation function layers, and the numbers are the same. The number of pooling layers is one less than that of the Conv convolutional layer and the Relu activation function layer. The number of fully connected layers is one. Along the data transmission direction, the first Conv convolutional layer is used to access the data output by the input layer. The data output by the second Relu activation function layer is transmitted to the first pooling layer, then to the third Conv convolutional layer. The last pooling layer is used to output the data to the fully connected layer. The fully connected layer is used to map the data to a 256-dimensional feature space and output it to the normalization layer. The normalization layer is used to normalize the face features to a range with a radius of 1.

4. The method according to claim 3, wherein, the convolution kernel size of each Conv convolutional layer is 3×3, and the convolution stride is 1.

5. The method according to claim 3, wherein, the following loss function is adopted during the training of the two-dimensional face recognition model: Floss = κ 1 F1 + κ 2 F2 Among them, κ 1 , κ 2 are the weight factors of the loss functions F1 and F2 respectively. L1 represents the maximum distance between the training face features of each face category and the center point feature. λ is a balance factor, which is in the range of 0.01 - 0.

1. xi represents the feature of the i-th category of training faces, pi represents the center point of the feature of the i-th category of training faces, W1 is the connection weight vector of the i-th neuron, θ is the connection offset of the i-th neuron, W2 is the connection weight vector of the j-th neuron, is the connection offset of the j-th neuron.

6. The method according to claim 2, wherein, after the step of obtaining the RGB depth image of the face area of the target face, before the step of inputting the face area RGB image information into the two-dimensional face recognition model for face recognition, smooth processing is performed on the RGB depth image of the face area.

7. A face recognition chip, wherein, it includes a storage unit and a processing unit. A computer program is stored in the storage unit. When the computer program is executed by the processing unit, the processing unit executes the steps of the method according to any one of claims 1 to 6.

8. One or more non-volatile readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method according to any one of claims 1 to 6.

Images