Deep learning-based three-dimensional face recognition method, system, medium, and device

Abstract

The application relates to a three-dimensional face recognition method, system, medium and equipment based on deep learning, which comprises the following steps: acquiring a first face depth image and a second face depth image; recognizing the first face depth image to obtain a first focusing area; recognizing the first face depth image to obtain a second focusing area; comparing the first focusing area with the second focusing area to determine whether the first focusing area and the second focusing area are completely coincident; and inputting the first face depth image and the second face depth image into a pre-established face recognition model to complete face recognition in the case that the first focusing area and the second focusing area are not completely coincident. The three-dimensional face recognition based on deep learning is realized by focusing on different parts of the face through twice face image collection with different focal lengths according to the depth information of the face, and by comparing the focusing areas of the two images.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, specifically a three-dimensional facial recognition method, system, medium, and device based on deep learning. Background Technology

[0002] Facial recognition is a biometric technology that identifies individuals based on their facial features. Compared to two-dimensional facial recognition, three-dimensional data is richer and provides more information. Furthermore, three-dimensional data lacks brightness information and is unaffected by lighting conditions; 3D geometric information is less susceptible to non-geometric changes in appearance, such as makeup; and facial expression changes are more easily processed using three-dimensional geometric information. Therefore, facial recognition based on three-dimensional models is currently the most effective method for addressing issues related to pose, lighting, and facial expression changes.

[0003] Existing 3D face recognition is basically based on 3D structured light, which has a relatively complex recognition principle and is highly dependent on hardware devices. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a deep learning-based 3D face recognition method that can solve the technical problems of existing 3D face recognition relying on 3D structured light hardware and having a complex recognition principle.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] The present invention provides a deep learning-based three-dimensional face recognition method, comprising the following steps:

[0007] Acquire a first face depth image and a second face depth image. The first face depth image is acquired by a depth-of-field lens at a first focal length, and the second face depth image is acquired by a depth-of-field lens at a second focal length.

[0008] The first face depth image is identified to obtain a first focus area; the second face depth image is identified to obtain a second focus area.

[0009] The first focus area is compared with the second focus area to determine whether the first focus area and the second focus area completely overlap. If the first focus area and the second focus area do not completely overlap, the first face depth image and the second face depth image are input into the pre-established face recognition model to complete face recognition.

[0010] In one embodiment of the present invention, acquiring a first face depth image and a second face depth image includes:

[0011] Obtain the initial distance between the depth-of-field lens and the human face;

[0012] Adjust the focal length of the depth-of-field lens according to the first distance, and obtain the first face depth-of-field image;

[0013] The first distance is adjusted according to a preset value to obtain the second distance;

[0014] Adjust the focal length of the depth-of-field lens according to the second distance, and obtain the second face depth-of-field image.

[0015] In one embodiment of the present invention, recognizing the first face depth image to obtain a first focus area includes:

[0016] Obtain the RGB values ​​of the first face depth image, and convert the first face depth image into a first grayscale image based on the RGB values;

[0017] Remove the light spots from the first grayscale image, and obtain the pixel X in the first grayscale image after removing the light spots. i GRAY(X) i );

[0018] Calculate the X of adjacent pixels i Pixel X i-1 The difference in grayscale between G i =GRAY(X i )-GRAY(X i-1 );

[0019] The grayscale difference G i When > α, the pixel X i As boundary points, boundary bands are generated based on the boundary points, where α is a preset grayscale difference threshold;

[0020] Obtain the width w of the boundary band. When the width w of the boundary band is less than β, the area within the boundary band is taken as the first focus area, where β is a preset width threshold.

[0021] In one embodiment of the present invention, recognizing the first face depth image to obtain a second focus area includes:

[0022] Obtain the RGB values ​​of the second face depth image, and convert the second face depth image into a second grayscale image based on the RGB values;

[0023] Remove the light spots from the second grayscale image, and obtain the grayscale value GRAY(X) of pixel Xi′ in the second grayscale image after removing the light spots. i );

[0024] Calculate the X of adjacent pixels i Pixel X i-1The difference in grayscale between G i =GRAY(X i )-GRAY(X i-1 );

[0025] The grayscale difference G i When > α, the pixel X i As boundary points, boundary bands are generated based on the boundary points, where α is a preset grayscale difference threshold;

[0026] Obtain the width w of the boundary band. When the width w of the boundary band is less than β, the area within the boundary band is taken as the second focus area, where β is a preset width threshold.

[0027] In one embodiment of the present invention, comparing the first focusing area with the second focusing area to determine whether the first focusing area and the second focusing area completely overlap includes:

[0028] Obtain the pixel A(x, y) of the first focus region and the pixel A′(x, y) of the second focus region;

[0029] When the coordinates of pixel A(x, y) and pixel A′(x, y) are the same, remove pixel A(x, y) and pixel A′(x, y);

[0030] If the number of pixels remaining in the first or second focus area after removing pixels is greater than a preset threshold, it is determined that the first focus area and the second focus area do not completely overlap; otherwise, it is determined that the first focus area and the second focus area completely overlap.

[0031] In one embodiment of the present invention, the process of establishing the face recognition model includes:

[0032] Obtain a face depth image containing the focus area, and construct a training dataset using the face depth image;

[0033] A face recognition model is obtained by training a pre-established artificial neural network based on the training dataset.

[0034] In one embodiment of the present invention, after recognizing the second face depth image, the method further includes:

[0035] When the second face depth image does not contain the focus area, the face corresponding to the first face depth image and the second face depth image is determined to be a planar image.

[0036] This invention also provides a deep learning-based 3D face recognition system, comprising:

[0037] The acquisition module is used to acquire a first face depth image and a second face depth image. The first face depth image is acquired by a depth-of-field lens at a first focal length, and the first face depth image is acquired by a depth-of-field lens at a second focal length.

[0038] The first recognition module is used to recognize the first face depth image to obtain a first focus area; and to recognize the second face depth image to obtain a second focus area.

[0039] The second recognition module is used to compare the first focus area with the second focus area, determine whether the first focus area and the second focus area completely overlap, and if the first focus area and the second focus area do not completely overlap, input the first face depth image and the second face depth image into the pre-established face recognition model to complete face recognition.

[0040] The present invention also provides a storage medium storing a computer program, which, when loaded and executed by a processor, implements the artificial intelligence commodity inspection method described above.

[0041] The present invention also provides an apparatus, comprising: a processor and a memory; wherein the memory is used to store a computer program; and the processor is used to load and execute the computer program to enable the apparatus to perform the artificial intelligence commodity inspection method as described above.

[0042] The beneficial effects of this invention are as follows: This invention provides a deep learning-based 3D face recognition method, system, medium, and device. It acquires a first face depth image and a second face depth image; identifies the first face depth image to obtain a first focus region; identifies the first face depth image to obtain a second focus region; compares the first and second focus regions to determine if they completely overlap; and if they do not completely overlap, inputs the first and second face depth images into a pre-established face recognition model to complete face recognition. Relying on the depth information of the face, and utilizing two face image acquisitions with different focal lengths, different parts of the face are focused on. By comparing the focus regions of the two images, deep learning-based 3D face recognition is achieved. Attached Figure Description

[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0044] Figure 1 This is a schematic diagram of the structure of the present invention;

[0045] Figure 2 This is a schematic diagram of the structure of the present invention; Detailed Implementation

[0046] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0047] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the layers related to the present invention and are not drawn according to the actual number, shape and size of the layers in the actual implementation. In the actual implementation, the form, number and proportion of each layer can be arbitrarily changed, and the layer layout may also be more complex.

[0048] Numerous details are explored in the following description to provide a more thorough explanation of embodiments of the invention; however, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details.

[0049] First, it's important to clarify that while existing facial recognition technology is widely used in internet payments, current smartphones, tablets, and other smart devices generally only support 2D facial payments. This involves capturing a 2D facial image through a camera and then using that image for payment. However, 2D facial payments present numerous security risks. Therefore, 3D facial recognition technology, such as 3D structured light, has emerged to enhance the security of facial payments. However, current 3D structured light technology places higher demands on the hardware of smart devices.

[0050] Therefore, in order to solve the above problems, such as Figure 1 As shown: This embodiment of a deep learning-based 3D face recognition method includes the following steps:

[0051] S110. Acquire a first face depth image and a second face depth image. The first face depth image is acquired by a depth-of-field lens at a first focal length, and the first face depth image is acquired by a depth-of-field lens at a second focal length.

[0052] In this embodiment, the face depth image is acquired using a depth of field (Tof) lens, and the vertical depth information of the face is used to focus at different focal lengths to obtain a first face depth image and a second face depth image.

[0053] S120. Recognize the first face depth image to obtain the first focus area; recognize the second face depth image to obtain the second focus area;

[0054] In step S120, since different focal lengths are used to capture the face, if the face is a real face rather than an image, both the first face depth image and the second face depth image should have a focus area, and the first focus area and the second focus area should not completely overlap.

[0055] S130. Compare the first focus area with the second focus area to determine whether the first focus area and the second focus area completely overlap. If the first focus area and the second focus area do not completely overlap, input the first face depth image and the second face depth image into the pre-established face recognition model to complete face recognition.

[0056] In step S130, if both the first focus area and the second focus area exist and do not completely overlap, it indicates that the face has depth information, and therefore a face recognition model can be used for recognition.

[0057] In one embodiment of the present invention, the process of acquiring the first face depth image and the second face depth image may include steps S210 to S240:

[0058] S210. Obtain the first distance between the depth-of-field lens and the face;

[0059] In this embodiment, the first distance is obtained by a pre-set ranging sensor, which can be an infrared ranging sensor or an ultrasonic ranging sensor, etc.

[0060] S220. Adjust the focal length of the depth-of-field lens according to the first distance and obtain the first face depth-of-field image;

[0061] In step S220, regardless of whether the depth-of-field lens captures a real human face or a planar image, it can be captured and focused using the corresponding focal length.

[0062] S230. Adjust the first distance according to the preset value to obtain the second distance;

[0063] In step S230, the preset value is generally the distance from the tip of the nose to the eye socket, usually around 8-10 cm.

[0064] S240. Adjust the focal length of the depth-of-field lens according to the second distance and obtain a second face depth-of-field image;

[0065] If the second face depth image acquired after adjusting the focus corresponds to a human face, then the focus should be on the area from the eye socket to the ear. If it is a planar image, then it cannot be focused.

[0066] In one embodiment of the present invention, the process of recognizing a first face depth image to obtain a first focus area may include steps S310 to S350:

[0067] S310. Obtain the RGB values ​​of the first face depth image, and convert the first face depth image into a first grayscale image based on the RGB values;

[0068] In this embodiment, before performing focus region recognition on the focus region of the first face depth image, the first face depth image needs to be preprocessed. In this embodiment, the image is preprocessed by converting it to a grayscale image to eliminate interference from color values.

[0069] S320. Remove the light spots from the first grayscale image and obtain the pixel X in the first grayscale image after removing the light spots. i GRAY(X) i );

[0070] In step S320, since the edge of the light spot is smooth and distinct even when it is out of focus, the light spot needs to be removed first. In this embodiment, the light spot area is removed by the gray value threshold method, that is, the part with the gray value greater than the gray value threshold is directly removed. If there is no light spot, the gray value can be directly obtained.

[0071] S330. Calculate the neighboring pixel X i Pixel X i-1 The difference in grayscale between G i =GRAY(X i )-GRAY(X i-1 );

[0072] In this embodiment, the boundary is determined by calculating the difference between adjacent pixels.

[0073] S340. The grayscale difference G... i When > α, the pixel X i As boundary points, boundary bands are generated based on the boundary points, where α is a preset grayscale difference threshold;

[0074] S350. Obtain the width w of the boundary band. When the width w of the boundary band is less than β, take the area within the boundary band as the first focus area, where β is a preset width threshold.

[0075] In this embodiment, if the image is taken in a defocused state, the boundary pixels will surround the object and have a large width. Therefore, the width of the boundary band is used to determine whether the image is in a defocused state.

[0076] In one embodiment of the present invention, the process of recognizing a first face depth image to obtain a second focus area may include steps S410 to S450:

[0077] S410. Obtain the RGB values ​​of the second face depth image, and convert the second face depth image into a second grayscale image based on the RGB values;

[0078] S420. Remove the light spots from the second grayscale image and obtain the pixel X in the second grayscale image after removing the light spots. i The grayscale value of 'GRAY(X) i );

[0079] S430. Calculate the neighboring pixel X i Pixel X i-1 The difference in grayscale between G i =GRAY(X i )-GRAY(X i-1 );

[0080] S440. The grayscale difference G... i When > α, the pixel X i As boundary points, boundary bands are generated based on the boundary points, where α is a preset grayscale difference threshold;

[0081] S450. Obtain the width w of the boundary band. When the width w of the boundary band is less than β, take the area within the boundary band as the second focus area, where β is a preset width threshold.

[0082] Similarly, in this embodiment, the difference between adjacent pixels is used to determine the boundary, and then the focus area and out-of-focus area in the second face depth image are determined by calculating the width of the boundary.

[0083] In one embodiment of the present invention, the process of comparing the first focus area and the second focus area to determine whether the first focus area and the second focus area completely overlap may include steps S510 to S530:

[0084] S510. Obtain the pixel A(x, y) of the first focus area and the pixel A′(x, y) of the second focus area;

[0085] S520. When the coordinates of pixel A(x, y) and pixel A′(x, y) are the same, remove pixel A(x, y) and pixel A′(x, y);

[0086] S530. When the number of pixels remaining in the first focus area or the second focus area after removing pixels is greater than a preset number threshold, it is determined that the first focus area and the second focus area do not completely overlap; otherwise, it is determined that the first focus area and the second focus area completely overlap.

[0087] In this embodiment, the removal is performed directly based on the coordinates of the pixels. If the first focus area and the second focus area do not completely overlap, there will be a lot of pixel residues. This principle is used to determine whether the first focus area and the second focus area completely overlap.

[0088] In one embodiment of the present invention, the process of establishing a face recognition model includes steps S610 to S620:

[0089] S610. Obtain a face depth image containing the focus area, and construct a training dataset using the face depth image;

[0090] S620. Train the pre-established artificial neural network based on the training dataset to obtain a face recognition model.

[0091] In one embodiment of the present invention, the process after recognizing the second face depth image further includes step S710:

[0092] S710. When the second face depth image does not contain a focus area, determine that the face corresponding to the first face depth image and the second face depth image is a planar image.

[0093] This invention discloses a deep learning-based 3D face recognition method. The method acquires a first face depth image and a second face depth image. It then identifies the first face depth image to obtain a first focus region and the second face depth image to obtain a second focus region. The first and second focus regions are compared to determine if they completely overlap. If they do not completely overlap, the first and second face depth images are input into a pre-established face recognition model to complete the face recognition. By relying on the depth information of the face and using two face image acquisitions with different focal lengths, different parts of the face are focused. By comparing the focus regions of the two images, deep learning-based 3D face recognition is achieved.

[0094] like Figure 2 As shown, the present invention also provides a deep learning-based 3D face recognition system, comprising:

[0095] The acquisition module is used to acquire a first face depth image and a second face depth image. The first face depth image is acquired through a depth-of-field lens at a first focal length, and the second face depth image is acquired through a depth-of-field lens at a second focal length.

[0096] The first recognition module is used to recognize the first face depth image to obtain the first focus area; and to recognize the second face depth image to obtain the second focus area.

[0097] The second recognition module is used to compare the first focus area and the second focus area to determine whether the first focus area and the second focus area completely overlap. If the first focus area and the second focus area do not completely overlap, the first face depth image and the second face depth image are input into the pre-established face recognition model to complete face recognition.

[0098] This invention discloses a deep learning-based 3D face recognition system. The system acquires a first face depth image and a second face depth image. It then identifies the first face depth image to obtain a first focus region and the second face depth image to obtain a second focus region. The system compares the first and second focus regions to determine if they completely overlap. If they do not completely overlap, the system inputs both the first and second face depth images into a pre-established face recognition model to complete the face recognition. By relying on the depth information of the face and using two face image acquisitions with different focal lengths, different parts of the face are focused. By comparing the focus regions of the two images, deep learning-based 3D face recognition is achieved.

[0099] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the methods in this embodiment.

[0100] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0101] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory so that the terminal performs any of the methods in this embodiment.

[0102] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0103] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0104] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0105] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A three-dimensional face recognition method based on deep learning, characterized in that, The method comprises the steps of: obtaining a first face depth image and a second face depth image, the first face depth image being captured by a depth lens at a first focal length, and the second face depth image being captured by the depth lens at a second focal length; identifying the first face depth image to obtain a first focus area; identifying the second face depth image to obtain a second focus area; comparing the first focus area and the second focus area to determine whether the first focus area and the second focus area are completely coincident, wherein the determination of whether the first focus area and the second focus area are completely coincident comprises: acquire the pixel points of the first focus area , the pixel points of the second focus area ; When the pixel point is consistent with the coordinate of the pixel point , the pixel point is removed. ​ when the number of remaining pixel points of the first focus area or the second focus area after removing the pixel points is greater than a preset number threshold, it is determined that the first focus area and the second focus area are not completely coincident; otherwise, it is determined that the first focus area and the second focus area are completely coincident; and in the case that the first focus area and the second focus area are not completely coincident, inputting the first face depth image and the second face depth image into a pre-established face recognition model to complete face recognition. 2.The three-dimensional face recognition method based on deep learning of claim 1, wherein: Obtaining a first face depth image and a second face depth image comprises: obtaining a first distance between a depth lens and a face; adjusting the focal length of the depth lens according to the first distance, and obtaining the first face depth image; adjusting the first distance according to a preset value to obtain a second distance; adjusting the focal length of the depth lens according to the second distance, and obtaining the second face depth image. 3.The three-dimensional face recognition method based on deep learning of claim 1, characterized in that: Identifying the first face depth image to obtain a first focus area comprises: obtaining the RGB value of the first face depth image, and converting the first face depth image into a first grayscale image according to the RGB value; remove the light spot in the first gray image, obtain the gray value of the pixel point in the first gray image after removing the light spot ;​ calculating a gray scale difference between adjacent pixel points , pixel points ;​ The gray scale difference value When the pixel point is a boundary point, and a boundary band is generated according to the boundary point, wherein is a preset gray scale difference threshold value; Get the width of the boundary band In the width of the boundary band At that time, the area within the boundary zone is taken as the first focusing area, in which This is the preset width threshold. 4.The three-dimensional face recognition method based on deep learning of claim 1, characterized in that: Identifying the second face depth image to obtain a second focus area comprises: obtaining the RGB value of the second face depth image, and converting the second face depth image into a second grayscale image according to the RGB value; remove the light spot in the second gray image, obtain the gray value of the pixel point in the second gray image after removing the light spot ;​ calculating a gray scale difference between adjacent pixel points , pixel points ;​ The gray scale difference value When the pixel point is a boundary point, and a boundary band is generated according to the boundary point, wherein is a preset gray scale difference threshold value; Get the width of the boundary band In the width of the boundary band At that time, the area within the boundary zone is designated as the second focus area, in which This is the preset width threshold. 5.The three-dimensional face recognition method based on deep learning of claim 1, characterized in that: The process of establishing the face recognition model comprises: obtaining a face depth image containing a focus area, and constructing a training data set with the face depth image; training a pre-established artificial neural network according to the training data set to obtain a face recognition model. 6.The three-dimensional face recognition method based on deep learning of claim 1, characterized in that: After identifying the second face depth image, it further comprises: when the second face depth image does not contain a focus area, determining that the face corresponding to the first face depth image and the second face depth image is a planar image.

7. A deep learning-based three-dimensional face recognition system, characterized by, It comprises: a collection module for obtaining a first face depth image and a second face depth image, the first face depth image being captured by a depth lens at a first focal length, and the second face depth image being captured by the depth lens at a second focal length; a first identification module for identifying the first face depth image to obtain a first focus area; identifying the second face depth image to obtain a second focus area; The second identification module is configured to compare the first focus area with the second focus area to determine whether the first focus area and the second focus area are completely overlapped, and specifically configured to: acquire the pixel points of the first focus area , the pixel points of the second focus area ; When the pixel point is consistent with the coordinate of the pixel point , the pixel point is removed with the pixel point . When the number of remaining pixels in the first focus area or the second focus area after removing the pixels is greater than a preset number threshold, it is determined that the first focus area and the second focus area are not completely overlapped; otherwise, it is determined that the first focus area and the second focus area are completely overlapped. In addition, the second identification module is further configured to, when the first focus area and the second focus area are not completely overlapped, input the first face depth image and the second face depth image into a pre-established face recognition model to complete face recognition.

8. A storage medium having stored therein a computer program, characterized in that The computer program is loaded and executed by the processor to implement the three-dimensional face recognition method based on deep learning in any one of claims 1 to 6.

9. An apparatus, comprising: The device comprises: a processor and a memory, wherein the memory is configured to store a computer program, and the processor is configured to load and execute the computer program to enable the device to implement the three-dimensional face recognition method based on deep learning in any one of claims 1 to 6.

Images