3D Facial Reconstruction Method, Device, Electronic Device and Medium

Abstract

The present invention relates to the field of artificial intelligence technology, and specifically provides a method, device, electronic device and medium for three-dimensional face reconstruction, aiming to solve the technical problem of low accuracy of face reconstruction corresponding to the existing face reconstruction methods. For this purpose, the three-dimensional face reconstruction method of the present invention includes the following steps: obtaining an original speckle infrared image containing a human face; performing face detection on the original speckle infrared image to obtain a face region; determining the matching degree between each pixel in the face region and a reference image, and obtaining a depth map from the face region by using the matching degree; obtaining a high-resolution three-dimensional face image based on the depth map. In this way, the accuracy of three-dimensional face reconstruction is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and specifically provides a method, device, electronic device and medium for three-dimensional face reconstruction. Background Art

[0002] In recent years, in face application scenarios such as face recognition and liveliness detection, it is necessary to use a structured light camera to perform three-dimensional reconstruction of the face to improve the accuracy of face recognition and liveliness detection.

[0003] However, the existing face reconstruction devices are far from accurately describing the fine features of the face in terms of accuracy. Many existing algorithms use a binocular reconstruction system, which can only reconstruct a rough contour of the face and is easily affected by external conditions such as illumination and lack of texture. Their reconstruction accuracy of the face is low and it is difficult to meet the actual needs.

[0004] Correspondingly, a new three-dimensional face reconstruction solution is needed in this field to solve the above problems. Summary of the Invention

[0005] In order to overcome the above defects, the present invention is proposed to provide a technical solution to solve or at least partially solve the problem of low face reconstruction accuracy corresponding to the existing face reconstruction methods. The present invention provides a method, device, electronic device and medium for three-dimensional face reconstruction.

[0006] In a first aspect, the present invention provides a method for three-dimensional face reconstruction, including the following steps: obtaining an original speckle infrared image containing a face; performing face detection on the original speckle infrared image to obtain a face region; determining the matching degree between each pixel in the face region and a reference image, and obtaining a depth map from the face region by using the matching degree; obtaining a high-resolution three-dimensional face image based on the depth map.

[0007] In one embodiment, the matching degree includes a matching score, a confidence level and a disparity difference. Determining the matching degree between each pixel in the face region and a reference image includes: establishing a first sliding window centered on each pixel in the face region, and determining a first disparity when the first sliding window slides on the reference image; calculating a matching score corresponding to the first disparity; obtaining the maximum matching score and the second maximum matching score from all the matching scores; calculating the confidence level based on the maximum matching score and the second maximum matching score.

[0008] In one embodiment, calculating the matching score corresponding to the first disparity includes: performing an exclusive OR operation on a first image within the sliding window and a second image corresponding to the sliding window on the reference image; determining the matching score based on the area ratio of the result of the exclusive OR operation in the first sliding window.

[0009] In one embodiment, determining the degree of match between each pixel in the face region and the reference image further includes: establishing a second sliding window centered on each pixel in the reference image, determining a second disparity when the second sliding window slides over the face region; and determining a disparity difference based on the first disparity and the second disparity.

[0010] In one embodiment, obtaining a depth map from the face region using the degree of match includes: determining whether the match score, confidence, and disparity difference between each pixel in the face region and the reference image all meet a first preset condition; if so, regarding the pixel as a valid point, and if not, regarding the pixel as an invalid point; and obtaining a depth map based on all the valid points.

[0011] In one embodiment, before obtaining a depth map based on all the valid points, it further includes: obtaining edge points of each valid point, and selecting invalid points from the edge points; establishing a third sliding window centered on the disparity of the invalid point, and calculating the match score between the invalid point and the reference image; and when the match score meets a second preset condition, regarding the invalid point as a valid point.

[0012] In one embodiment, before performing face detection on the original speckle infrared image to obtain a face region, it further includes: performing an enhancement operation on the original speckle infrared image using a single-scale retina enhancement algorithm; performing binarization processing on the enhanced original speckle infrared image to obtain a binary speckle image; and removing noise in the binary speckle image.

[0013] In a second aspect, the present invention provides a face three-dimensional reconstruction device, including: an acquisition module configured to acquire an original speckle infrared image containing a face; a detection module configured to perform face detection on the original speckle infrared image to obtain a face region; a matching module configured to determine the degree of match between each pixel in the face region and the reference image, and obtain a depth map from the face region using the degree of match; and a determination module configured to obtain a high-resolution face three-dimensional image based on the depth map.

[0014] In a third aspect, there is provided an electronic device, which includes a processor and a storage device. The storage device is adapted to store multiple program codes, and the program codes are adapted to be loaded and run by the processor to execute the face three-dimensional reconstruction method described in any one of the foregoing items.

[0015] In a fourth aspect, there is provided a computer-readable storage medium, which stores multiple program codes therein, and the program codes are adapted to be loaded and run by a processor to execute the face three-dimensional reconstruction method described in any one of the foregoing items.

[0016] One or more of the above technical solutions of the present invention have at least one or more of the following beneficial effects:

[0017] The present invention provides a method for three-dimensional face reconstruction. First, an original speckle infrared image containing a human face is obtained. Then, face detection is performed on the original speckle infrared image to obtain a face region. Next, the matching degree between each pixel in the face region and a reference image is determined, and a depth map is obtained from the face region using the matching degree. Finally, a high-resolution three-dimensional face image is obtained based on the depth map. In this way, after detecting the face region, the matching degree between the face region and a reference image with higher accuracy is further measured using the matching score, confidence, and disparity difference, so as to obtain a depth map with higher accuracy, and finally a high-resolution three-dimensional face image is obtained, improving the accuracy of three-dimensional face reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Referring to the accompanying drawings, the disclosure of the present invention will become more understandable. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the protection scope of the present invention. In addition, similar numbers in the figures are used to represent similar components, where:

[0019] Figure 1 is a schematic diagram of the main step flow of a method for three-dimensional face reconstruction according to an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of the complete flow of a method for three-dimensional face reconstruction according to an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of the main structural block diagram of a three-dimensional face reconstruction device according to an embodiment of the present invention.

[0022] LIST OF REFERENCE NUMERALS:

[0023] 11: Acquisition module; 12: Detection module; 13: Matching module; 14: Determination module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] The following describes some embodiments of the present invention with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principle of the present invention and are not intended to limit the protection scope of the present invention.

[0025] In the description of the present invention, "module" and "processor" may include hardware, software, or a combination of both. A module may include a hardware circuit, various appropriate sensors, communication ports, memories, and may also include a software part, such as program code, or may be a combination of software and hardware. A processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other appropriate processor. The processor has data and / or signal processing functions. The processor may be implemented in software, in hardware, or in a combination of both. A non-transitory computer-readable storage medium includes any appropriate medium that can store program code, such as magnetic disks, hard disks, optical discs, flash memories, read-only memories, random access memories, and so on. The term "A and / or B" represents all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one of A or B" or "at least one of A and B" has a meaning similar to "A and / or B" and may include only A, only B, or A and B. The singular terms "a" and "the" may also include the plural form.

[0026] Currently, traditional face reconstruction devices far from accurately describe the subtle features of the face in terms of accuracy. Many existing algorithms use binocular reconstruction systems, which can only reconstruct the rough outline of the face and are easily affected by external conditions such as lighting and lack of texture. Their reconstruction accuracy of the face is low and it is difficult to meet the actual needs. For this reason, the present application provides a three-dimensional face reconstruction method, device, electronic device, and medium. First, an original speckle infrared image containing a face is obtained. Then, face detection is performed on the original speckle infrared image to obtain a face region. Secondly, the matching degree between each pixel in the face region and a reference image is determined, and a depth map is obtained from the face region using the matching degree. Finally, a high-resolution three-dimensional face image is obtained based on the depth map. In this way, after the face region is obtained by face detection, the matching degree between the face region and a reference image with higher accuracy is further measured using the matching score, confidence, and disparity difference, so as to obtain a depth map with higher accuracy, and finally a high-resolution three-dimensional face image is obtained, improving the accuracy of three-dimensional face reconstruction.

[0027] Refer to the attached Figure 1 , Figure 1 is a schematic diagram of the main step flow of a three-dimensional face reconstruction method according to an embodiment of the present invention. As Figure 1 shown, the three-dimensional face reconstruction method in the embodiment of the present invention mainly includes the following steps S101-step S104.

[0028] Step S101: Obtain an original speckle infrared image containing a face. Specifically, the original speckle infrared image containing a face in this step can be obtained by an infrared camera.

[0029] Step S102: Perform face detection on the original speckle infrared image to obtain the face region.

[0030] Specifically, use the multi-task convolutional neural network (MTCNN) algorithm after independent training to detect the original speckle infrared image, and detect the face region.

[0031] The MTCNN algorithm consists of three network structures: P-Net, R-Net, and O-Net. Among them, P-Net (Proposal Network) mainly obtains the candidate window and the bounding box regression vector of the face region. R-Net (Refine Network) obtains a more accurate candidate window based on P-Net. O-Net (Output Network) further improves the window accuracy based on R-Net and outputs the coordinates of 5 key points at the same time. The 5 key point coordinates include the coordinates of the left eye, right eye, nose tip, left mouth corner, and right mouth corner.

[0032] Step S103: Determine the matching degree between each pixel in the face region and the reference image, and use the matching degree to obtain the depth map from the face region.

[0033] When the camera leaves the factory, let the speckle project onto a plane parallel to the camera imaging plane and at a specified distance, and use the infrared speckle image captured by the infrared camera as the reference image. The reference image involved in this application is a standard binary image captured and processed when the camera leaves the factory. This image has high precision and can be used for later matching and optimization with the face region.

[0034] In order to optimize the efficiency of face reconstruction, this application first uses the grid method to screen pixels from the face region. That is, from each 10×10 grid in the face region, select a pixel in the upper left corner to calculate the matching degree. In this way, compared with calculating the matching degree for each pixel in the face region, the overall calculation amount is reduced by 100 times after grid screening, which is beneficial to improving the efficiency of 3D face reconstruction.

[0035] After screening out pixels using the grid method, perform pixel matching between each screened pixel in the face region and the reference image. Specifically, delete the pixels with lower matching degrees from the face region, so as to obtain a depth map with a higher matching degree with the reference image.

[0036] In a specific embodiment, the matching degree includes a matching score, a confidence level, and a parallax difference. Determining the matching degree between each pixel in the face region and the reference image includes: establishing a first sliding window centered on each pixel in the face region, and determining a first parallax when the first sliding window slides on the reference image; calculating a matching score corresponding to the first parallax; obtaining the maximum matching score and the second maximum matching score from all the matching scores; and calculating the confidence level based on the maximum matching score and the second maximum matching score.

[0037] In this application, the sliding window method is used. For each pixel selected from the face region, a rectangular window is established centered on it, and the rectangular window slides horizontally in the corresponding interval of the reference image. During the sliding process, multiple first parallaxes will be generated. The first parallax is the coordinate difference between the current pixel on the reference image and the central pixel point of the rectangular window. Therefore, during the process of the rectangular window or the sliding window sliding on the reference image, multiple first parallaxes will be generated.

[0038] In a specific embodiment, calculating the matching score corresponding to the first parallax includes: performing an exclusive OR operation on the first image within the sliding window and the second image corresponding to the sliding window on the reference image; and determining the matching score based on the area ratio of the result of the exclusive OR operation in the area of the first sliding window.

[0039] For each first parallax, the matching score corresponding to the first parallax can be calculated. Specifically, the exclusive OR (XOR) matching calculation method is adopted to calculate the matching score.

[0040] Specifically, an exclusive OR operation is performed on the image located within the sliding window in the face region and the image corresponding to the sliding window when it slides on the reference image. It can also be understood as performing an exclusive OR operation on the pixels at the same position on each image.

[0041] Exemplarily, a 3*3 sliding window is used for illustration. For the pixels at 9 positions in the sliding window, exclusive OR operations are performed on the pixels at the corresponding positions of the second image corresponding to the sliding window when it slides on the reference image, and the exclusive OR operation results at each pixel position in the sliding window are obtained. Generally speaking, the results of the exclusive OR operation are 0 and 1. Then, the area where the exclusive OR operation result is 1 among all pixel positions in the sliding window is statistically calculated, and the area ratio of this area in the entire sliding window is the matching score.

[0042] After calculating the matching scores of each pixel under all first parallaxes, the maximum matching score and the second maximum matching score can be selected from them, and the confidence level is calculated using the following formula:

[0043]

[0044] Among them, Conf(x,y) represents the confidence level, maxf(x,y,D(x,y represents the maximum matching score, and maxf(x,y,d) d≠D(x,y) represents the second maximum matching score.

[0045] In a specific embodiment, determining the matching degree between each pixel in the face region and the reference image further includes: establishing a second sliding window centered on each pixel in the reference image, determining the second parallax when the second sliding window slides on the face region; and determining the parallax difference based on the first parallax and the second parallax.

[0046] In the foregoing embodiment, the sliding window is established centered on the pixel points in the face region and slides on the reference image. In this embodiment, specifically, a sliding window is established centered on the pixels in the reference image and slides on the face region. When the sliding window slides, the second parallax is obtained by the difference between the coordinates of the current pixel on the face region and the coordinates of the central pixel point of the rectangular window. Then, the parallax difference is obtained by the difference between the first parallax and the second parallax of the pixels at the corresponding positions. The constraint of the parallax difference on the image is also the left-right consistency constraint.

[0047] In a specific embodiment, obtaining the depth map from the face region using the matching degree includes: determining whether the matching score, confidence level, and parallax difference between each pixel in the face region and the reference image all meet the first preset condition; if so, regarding the pixel as a valid point, and if not, regarding the pixel as an invalid point; and obtaining the depth map based on all valid points.

[0048] By simultaneously determining whether the matching score, confidence level, and parallax difference between each pixel in the face region and the reference image all meet the first preset condition, valid points are selected from the face region.

[0049] The first preset condition is that the matching score is greater than the matching score threshold, the confidence level is greater than the confidence level threshold, and the absolute value of the parallax difference is less than the parallax threshold. Those skilled in the art can understand that the matching score threshold, confidence level threshold, and parallax threshold can be obtained through experiments in advance. Through the confidence level, incorrect matches at discontinuous edges in the face region can be excluded. Through the parallax difference, that is, the left-right consistency constraint, incorrect matches in the occluded regions in the face region can be effectively removed to screen a depth map with higher accuracy from the face region.

[0050] Through the first preset condition, the pixels that meet the conditions are screened out from the face region as valid points, and the pixel points that do not meet the conditions are regarded as invalid points.

[0051] The number of valid points screened out through the foregoing first preset condition may be small. Therefore, the present application can perform secondary screening on the invalid points around the screened valid points to obtain more valid points.

[0052] In a specific embodiment, before obtaining the depth map based on all valid points, it further includes: obtaining the edge points of each valid point and selecting invalid points from the edge points; establishing a third sliding window centered on the disparity of the invalid points, and calculating the matching score between the invalid points and the reference image; in the case where the matching score meets the second preset condition, taking the invalid points as valid points.

[0053] Use the iterative growth method to perform a secondary screening on the invalid points around the valid points. The specific process is as follows: First, search for the edge points of all valid points and add them to the queue. Take out the edge points of each valid point from the queue in turn, and take out the invalid points from the edge points. Then, take the disparity corresponding to the maximum matching score of the invalid point as the center point, and recalculate the matching score between the invalid point and the reference image. If the matching score is greater than the second preset condition, take the invalid point as a valid point. Where the second preset condition is that the matching score between the invalid point and the reference image is greater than the matching score threshold. This process can be repeatedly executed until all the edge points in the queue have been processed.

[0054] In addition, after obtaining all the valid points, the valid points are the valid pixels on the face area, and interpolation can also be performed on all the valid pixels. Specifically, for each valid pixel, the disparity corresponding to the maximum matching score of the valid pixel is d, and the maximum matching scores corresponding to the disparities of d + 1 and d - 1 are extracted for interpolation. In this way, the number of valid pixels can be further enriched, thereby obtaining a more accurate depth map.

[0055] Step S104: Obtain a high-resolution 3D face image based on the depth map.

[0056] After obtaining the depth map based on the foregoing steps, using the camera internal and external parameter matrix transformation formula, the depth map can be converted into a 3D face image. Since the depth map has a higher resolution than the original speckle infrared image, the converted 3D face image is a high-resolution 3D face image.

[0057] Finally, the depth map and the high-resolution 3D face image can be output simultaneously. Specifically, the depth map is output in the 16-bit PNG image format, and the high-resolution 3D face image is output in the PLY format.

[0058] Based on the above steps S101 - S104, first, an original speckle infrared image containing a human face is obtained. Then, face detection is performed on the original speckle infrared image to obtain a face region. Next, the matching degree between each pixel in the face region and a reference image is determined, and a depth map is obtained from the face region using the matching degree. Finally, a high - resolution 3D face image is obtained based on the depth map. In this way, after face detection to obtain the face region, the matching degree between the face region and a reference image with higher precision is further measured using the matching score, confidence, and disparity difference, so as to obtain a depth map with higher precision, and finally obtain a high - resolution 3D image, improving the accuracy of 3D face reconstruction.

[0059] In a specific embodiment, before performing face detection on the original speckle infrared image to obtain a face region, it further includes: performing an enhancement operation on the original speckle infrared image using a single - scale retina enhancement algorithm; performing binarization processing on the enhanced original speckle infrared image to obtain a binary speckle image; and removing noise in the binary speckle image.

[0060] The process of performing an enhancement operation on the original speckle infrared image using a single - scale retina enhancement algorithm (SSR) includes: First, for the input original speckle infrared image I(x, y), a Gaussian surrounding space constant is specified as the scale, and the image after Gaussian blur operation on the original speckle infrared image according to the specified scale is L(x, y). Calculate the value of Log[R(x, y)] according to log(R) = log(I) - log(L). Then, quantize Log[R(x, y)] into pixel values in the range of 0 to 255, so as to obtain a speckle infrared image with enhanced edge information. Through the single - scale retina enhancement algorithm, the edge information of the speckle infrared image can be better enhanced, and the integrity of the speckles can be increased. In addition, through the single - scale retina enhancement algorithm, on the premise of trying to prevent the increase of image noise and over - exposure, the speckle quality and clarity can be maximally improved.

[0061] In addition, in addition to the single - scale retina enhancement algorithm (SSR), adaptive histogram equalization can also be used to perform an enhancement operation on the original speckle infrared image.

[0062] The infrared speckle image can also be processed into a binary speckle image through adaptive binarization, and salt - and - pepper noise in the binary speckle image can be removed through connected - component denoising and mean filtering.

[0063] It should be noted that although the above - mentioned embodiments describe the various steps in a specific order, those skilled in the art can understand that in order to achieve the effects of the present invention, the different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the protection scope of the present invention.

[0064] Exemplarily, it can be implemented according to Figure 2 the process shown, specifically as shown in the following steps S201 to S206.

[0065] Step S201: Obtain the original speckle infrared image.

[0066] Step S202: Optimize the speckle quality of the original speckle infrared image using the single-scale retinex algorithm (SSR).

[0067] Step S203: Preprocess the speckle infrared image with optimized speckle quality. The preprocessing can be adaptive binarization processing, connected component noise reduction processing, mean filtering, etc., but is not limited thereto.

[0068] Step S204: Perform face detection to obtain the face region.

[0069] Step S205: Perform 3D face reconstruction on the basis of the face region to obtain a depth map, and finally convert the depth map into a high-resolution 3D face image.

[0070] Step S206: Output the high-resolution 3D face image.

[0071] In this way, the accuracy of 3D face reconstruction is improved.

[0072] Furthermore, the present invention also provides a 3D face reconstruction device. Refer to the attached Figure 3 , Figure 3 which is the main structural block diagram of the 3D face reconstruction device according to an embodiment of the present invention.

[0073] As Figure 3 shown, the 3D face reconstruction device in the embodiment of the present invention mainly includes an acquisition module 11, a detection module 12, a matching module 13, and a determination module 14. In some embodiments, one or more of the acquisition module 11, the detection module 12, the matching module 13, and the determination module 14 can be combined together into one module. In some embodiments, the acquisition module 11 can be configured to acquire the original speckle infrared image containing a face. The detection module 12 can be configured to perform face detection on the original speckle infrared image to obtain the face region. The matching module 13 can be configured to determine the matching degree between each pixel in the face region and the reference image, and obtain the depth map from the face region using the matching degree. The determination module 14 can be configured to obtain a high-resolution 3D face image based on the depth map. In one implementation manner, the description of the specific implementation functions can be referred to the steps S101 - S104.

[0074] The above 3D face reconstruction device is used to execute Figure 1The embodiments of the 3D face reconstruction method shown have similar technical principles, technical problems solved, and technical effects. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process and related explanations of the 3D face reconstruction device can refer to the content described in the embodiments of the 3D face reconstruction method, which will not be elaborated here.

[0075] In addition, the 3D face reconstruction device of the present invention can be deployed on a general-purpose CPU of a mobile device and achieve real-time reconstruction effects.

[0076] Those skilled in the art can understand that all or part of the processes in the method of an embodiment of the present invention can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable storage medium can include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0077] Furthermore, the present invention also provides an electronic device. In an embodiment of the electronic device according to the present invention, the electronic device includes a processor and a storage device. The storage device can be configured to store a program for executing the 3D face reconstruction method of the above method embodiment, and the processor can be configured to execute the program in the storage device. The program includes, but is not limited to, the program for executing the 3D face reconstruction method of the above method embodiment. For the convenience of description, only the parts related to the embodiments of the present invention are shown. For the specific technical details not disclosed, please refer to the method part of the embodiments of the present invention.

[0078] Furthermore, the present invention also provides a computer-readable storage medium. In an embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium may be configured to store a program for executing the face three-dimensional reconstruction method in the above method embodiment. This program can be loaded and run by a processor to implement the above face three-dimensional reconstruction method. For ease of description, only the parts related to the embodiments of the present invention are shown. For the specific technical details not disclosed, please refer to the method part of the embodiments of the present invention. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0079] Furthermore, it should be understood that since the setting of each module is only for illustrating the functional units of the device of the present invention, the corresponding physical devices of these modules can be the processor itself, or a part of the software in the processor, a part of the hardware, or a part of the combination of software and hardware. Therefore, the number of each module in the figure is only illustrative.

[0080] Those skilled in the art can understand that the various modules in the device can be adaptively split or combined. Such splitting or combination of specific modules will not cause the technical solution to deviate from the principle of the present invention. Therefore, the technical solutions after splitting or combination will all fall within the protection scope of the present invention.

[0081] So far, the technical solution of the present invention has been described in conjunction with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

Claims

1. A three-dimensional face reconstruction method, characterized in that, It includes the following steps: Obtain an original speckle infrared image containing a human face; Perform face detection on the original speckle infrared image to obtain a face region; Determine the matching degree between each pixel in the face region and a reference image, and obtain a depth map from the face region by using the matching degree; Obtain a high-resolution three-dimensional face image based on the depth map; The matching degree includes a matching score, a confidence level, and a disparity difference. Determining the matching degree between each pixel in the face region and the reference image includes: Establish a first sliding window centered on each pixel in the face region, and determine a first disparity when the first sliding window slides on the reference image; Calculate the matching score corresponding to the first disparity; Obtain the maximum matching score and the second maximum matching score from all the matching scores; Calculate the confidence level based on the maximum matching score and the second maximum matching score.

2. The three-dimensional face reconstruction method according to claim 1, characterized in that Calculating the matching score corresponding to the first disparity includes: Perform an exclusive OR operation on the first image within the sliding window and the second image corresponding to the sliding window on the reference image; Determine the matching score based on the area ratio of the exclusive OR operation result within the first sliding window.

3. The 3D face reconstruction method according to claim 1, characterized in that, Determining the matching degree between each pixel in the face region and the reference image further includes: Establish a second sliding window centered on each pixel in the reference image, and determine a second disparity when the second sliding window slides on the face region; Determine the disparity difference based on the first disparity and the second disparity.

4. The three-dimensional face reconstruction method according to claim 1, wherein Obtaining a depth map from the face region by using the matching degree includes: Judge whether the matching score, the confidence level, and the disparity difference between each pixel in the face region and the reference image all meet a first preset condition; If so, regard the pixel as a valid point, and if not, regard the pixel as an invalid point; Obtain a depth map based on all the valid points.

5. The three-dimensional face reconstruction method according to claim 4, wherein Before obtaining a depth map based on all the valid points, it further includes: Obtain the edge points of each valid point, and select the invalid points from the edge points; Establish a third sliding window centered on the disparity of the invalid point, and calculate the matching score between the invalid point and the reference image; When the matching score meets a second preset condition, regard the invalid point as a valid point.

6. The three-dimensional face reconstruction method according to claim 1, wherein Before performing face detection on the original speckle infrared image to obtain a face region, it further includes: Perform an enhancement operation on the original speckle infrared image by using a single-scale retina enhancement algorithm; Perform binarization processing on the original speckle infrared image after the enhancement operation to obtain a binary speckle map; and Perform a noise removal operation on the binary speckle map.

7. A three-dimensional face reconstruction device, characterized in that, It includes: An acquisition module, configured to obtain an original speckle infrared image containing a human face; A detection module, configured to perform face detection on the original speckle infrared image to obtain a face region; A matching module, configured to determine a matching degree between each pixel in the face region and a reference image, and obtain a depth map from the face region by using the matching degree, where the matching degree includes a matching score, a confidence level, and a disparity difference, and the matching module is further configured to: establish a first sliding window centered on each pixel in the face region, and determine a first disparity when the first sliding window slides on the reference image; calculate a matching score corresponding to the first disparity; obtain a maximum matching score and a sub-maximum matching score from all the matching scores; calculate a confidence level based on the maximum matching score and the sub-maximum matching score; A determination module, configured to obtain a high-resolution three-dimensional face image based on the depth map.

8. An electronic device, comprising a processor and a storage device, wherein the storage device is adapted to store multiple program codes, characterized in that The program code is adapted to be loaded and run by the processor to execute the three-dimensional face reconstruction method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing multiple program codes, characterized in that, The program code is adapted to be loaded and run by a processor to execute the three-dimensional face reconstruction method according to any one of claims 1 to 6.

Images