A face recognition method and device

Abstract

The embodiment of the present invention discloses a face recognition method and device, which includes: obtaining face images collected by multiple image acquisition devices in the same acquisition cycle, wherein the multiple image acquisition devices shoot a target face from different shooting angles, wherein part of the target face is blocked; detecting facial feature points of unblocked parts of the target face from the multiple face images; determining three-dimensional position information corresponding to the target facial feature points in the target face based on the facial feature points detected in each face image and device information of each image acquisition device; determining a target three-dimensional face model corresponding to the target face based on an initial three-dimensional face model, preset constraints and the three-dimensional position information corresponding to the target facial feature points, wherein the target three-dimensional face model includes the three-dimensional position information corresponding to the facial feature points of the blocked parts of the target face, so as to realize accurate recognition of the partially blocked face.

Description

Technical Field

[0001] The present invention relates to the field of face recognition technology, and in particular to a face recognition method and device. Background Art

[0002] With the continuous development of technology, face recognition technology is becoming more and more mature, and the accuracy of its recognition results is getting higher and higher. At present, in related face recognition technologies, in order to ensure the accuracy of face recognition results, it is generally necessary to obtain an image containing a complete and unobstructed face, and then detect and recognize the image to obtain a face recognition result.

[0003] In some scenarios, it is inevitable that a person's face is partially blocked, such as when a person wears a mask or sunglasses. In the related technology, the person is generally required to cooperate in removing objects blocking the face, such as requiring the person to remove the mask or sunglasses. If the person does not remove the objects blocking the face, the related face recognition technology cannot obtain a highly accurate face recognition result.

[0004] Therefore, how to provide a method for accurately identifying partially occluded faces becomes an urgent problem to be solved. Summary of the invention

[0005] The present invention provides a face recognition method and device to realize accurate recognition of a partially obscured face.

[0006] The specific technical solutions are as follows:

[0007] In a first aspect, an embodiment of the present invention provides a face recognition method, the method comprising:

[0008] Obtaining facial images captured by multiple image acquisition devices in the same acquisition cycle, wherein the multiple image acquisition devices capture the target face from different shooting angles, and part of the target face is blocked;

[0009] Detecting facial feature points of unobstructed parts of the target face from the multiple facial images;

[0010] Determine the three-dimensional position information corresponding to the target facial feature points in the target face based on the facial feature points detected in each facial image and the device information of each image acquisition device;

[0011] Based on the initial three-dimensional face model, preset constraints and the three-dimensional position information corresponding to the target facial feature points, the target three-dimensional face model corresponding to the target face is determined, wherein the target three-dimensional face model includes the three-dimensional position information corresponding to the facial feature points of the occluded parts of the target face, and the initial three-dimensional face model includes: multiple sample face models and their corresponding weight parameters, and the preset constraints are used to constrain the values ​​of the weight parameters corresponding to the multiple sample face models.

[0012] Optionally, the step of determining a target three-dimensional face model corresponding to the target face based on the initial three-dimensional face model, preset constraints and the three-dimensional position information corresponding to the target face feature points includes:

[0013] Based on the three-dimensional position information corresponding to the first reference facial feature point corresponding to each target facial feature point in each reference facial feature point in the initial three-dimensional facial model and the device information of each image acquisition device, determine the first projection position information corresponding to each first reference facial feature point in each facial image, wherein the reference facial feature points include: facial feature points of various parts of the face, and the same parts of the face corresponding to the first reference facial feature points and the target facial feature points that have a corresponding relationship;

[0014] Determine second projection position information corresponding to each target facial feature point in each facial image based on the three-dimensional position information corresponding to the target facial feature point and the device information of each image acquisition device;

[0015] Based on the second projection position information corresponding to each target facial feature point, the first projection position information corresponding to each first reference facial feature point and preset constraints, a target three-dimensional facial model corresponding to the target face is determined.

[0016] Optionally, the step of determining a target three-dimensional face model corresponding to the target face based on the second projection position information corresponding to each target face feature point, the first projection position information corresponding to each first reference face feature point, and preset constraints includes:

[0017] Determine a first projection error corresponding to the target face based on the second projection position information corresponding to each target facial feature point, the first projection position information corresponding to each first reference facial feature point, and a preset constraint condition;

[0018] Determine whether the first projection error does not exceed a preset error threshold;

[0019] If it is determined that the first projection error does not exceed a preset error threshold, determining the current initial three-dimensional face model as a target three-dimensional face model corresponding to the target face;

[0020] If it is determined that the first projection error exceeds a preset error threshold, adjusting current values ​​of weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain a current initial three-dimensional face model;

[0021] Determine new first projection position information corresponding to each first reference facial feature point in each facial image based on new three-dimensional position information corresponding to the first reference facial feature point in the current initial three-dimensional facial model and device information of each image acquisition device;

[0022] Based on the second projection position information corresponding to each target facial feature point, the new first projection position information corresponding to each first reference facial feature point and the preset constraints, determine the first projection error corresponding to the target face; return to the step of determining whether the first projection error does not exceed the preset error threshold.

[0023] Optionally, the step of determining a target three-dimensional face model corresponding to the target face based on the initial three-dimensional face model, preset constraints and the three-dimensional position information corresponding to the target face feature points includes:

[0024] Based on the current values ​​of the multiple sample face models and their corresponding weight parameters in the initial three-dimensional face model, determine the three-dimensional position information corresponding to each second reference face feature point corresponding to the target face feature point, wherein the second reference face feature point and the target face feature point that have a corresponding relationship correspond to the same part of the face;

[0025] Determine a first distance error corresponding to the target face based on the three-dimensional position information corresponding to each target facial feature point and the three-dimensional position information corresponding to each corresponding second reference facial feature point and preset constraints;

[0026] Determine whether a first distance error corresponding to the target face does not exceed a first distance error threshold;

[0027] If it is determined that the first distance error does not exceed a first distance error threshold, determining the current initial three-dimensional face model as a target three-dimensional face model corresponding to the target face;

[0028] If it is determined that the first distance error exceeds a first distance error threshold, adjusting current values ​​of weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain a current initial three-dimensional face model;

[0029] Based on the adjusted values ​​of the multiple sample face models and their corresponding weight parameters in the current initial three-dimensional face model, determining new three-dimensional position information corresponding to each second reference face feature point;

[0030] Based on the three-dimensional position information corresponding to each target facial feature point and the new three-dimensional position information corresponding to each corresponding second reference facial feature point and the preset constraints, determine the first distance error corresponding to the target face; return to the step of determining whether the first distance error corresponding to the target face does not exceed the first distance error threshold.

[0031] Optionally, the preset constraint condition includes a preset regularization constraint condition and a preset hyperparameter;

[0032] Before the step of determining a target 3D face model corresponding to the target face based on the initial 3D face model, the preset constraints and the 3D position information corresponding to the target face feature points, the method further includes:

[0033] The process of obtaining a preset hyperparameter in a preset constraint condition includes:

[0034] Obtaining real three-dimensional position information corresponding to the debugging face feature points of multiple debugging faces, wherein the debugging face feature points of each debugging face include facial feature points of various parts of the face;

[0035] Obtaining each initial constraint condition, wherein different initial constraints include the same preset regularization constraint condition and different initial hyperparameters;

[0036] For each initial constraint condition, perform the following steps AH to determine a preset constraint condition from the initial constraint condition, and determine a preset hyperparameter in the preset constraint condition;

[0037] Step A: for each debugging face, based on the three-dimensional position information corresponding to the third reference facial feature point at the designated position in the initial three-dimensional facial model, the three-dimensional position information corresponding to the designated debugging facial feature point at the designated position in the debugging facial feature points of the debugging face, and the preset regular constraint condition and initial hyperparameters included in the initial constraint condition, determine the second distance error corresponding to the debugging face, wherein the designated position is the same position as the position of the unobstructed part of the target face;

[0038] Step B: for each debugging face, determining whether the second distance error corresponding to the debugging face does not exceed the second distance error threshold;

[0039] Step C: if it is determined that the second distance error corresponding to the debugging face exceeds the second distance error threshold, adjusting the current values ​​of the weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain the current initial three-dimensional face model;

[0040] Step D: determining new three-dimensional position information corresponding to each third reference facial feature point based on current values ​​of weight parameters corresponding to the multiple sample facial models in the current initial three-dimensional facial model;

[0041] Step E: determining a second distance error corresponding to the debugging face based on the three-dimensional position information corresponding to the designated debugging face feature point at the designated position among the debugging face feature points of the debugging face, the new three-dimensional position information corresponding to the third reference face feature point, and the preset regular constraint condition and initial hyperparameter included in the initial constraint condition; returning to step B;

[0042] Step F: if it is determined that the second distance error corresponding to the debugging face does not exceed the second distance error threshold, the current initial three-dimensional face model is determined as the debugging three-dimensional face model corresponding to the debugging face;

[0043] Step G: determining the debugging three-dimensional position information corresponding to the debugging face feature points at other positions outside the specified position in the debugging face based on the current values ​​of the multiple sample face models and their corresponding weight parameters in the debugging three-dimensional face model corresponding to the debugging face;

[0044] Step H: determining the three-dimensional position error of the debugging face corresponding to the initial constraint condition based on the debugging three-dimensional position information corresponding to the debugging face feature points at other positions outside the specified position in the debugging face and the real three-dimensional position information;

[0045] Based on the three-dimensional position error of each debugging face corresponding to each initial constraint condition, a preset constraint condition is determined from the initial constraint condition, and its initial hyperparameter is determined as the preset hyperparameter, wherein the initial constraint condition with the smallest sum of the three-dimensional position errors of the corresponding debugging faces is determined as the preset constraint condition; or, the initial constraint condition with the smallest number of three-dimensional position errors of the corresponding debugging faces is determined as the preset constraint condition.

[0046] Optionally, the target face is a face wearing a mask, and the unobstructed portion is a portion of the target face that is not obstructed by the mask.

[0047] In a second aspect, an embodiment of the present invention provides a face recognition device, the device comprising:

[0048] A first acquisition module is configured to obtain face images acquired by multiple image acquisition devices in the same acquisition cycle, wherein the multiple image acquisition devices photograph the target face from different shooting angles, and part of the target face is blocked;

[0049] A detection module is configured to detect facial feature points of unobstructed parts of a target face from the multiple face images;

[0050] A first determination module is configured to determine the three-dimensional position information corresponding to the target facial feature points in the target face based on the facial feature points detected in each facial image and the device information of each image acquisition device;

[0051] The second determination module is configured to determine the target three-dimensional face model corresponding to the target face based on the initial three-dimensional face model, preset constraints and the three-dimensional position information corresponding to the target face feature points, wherein the target three-dimensional face model includes the three-dimensional position information corresponding to the facial feature points of the occluded parts of the target face, the initial three-dimensional face model includes: multiple sample face models and their corresponding weight parameters, and the preset constraints are used to constrain the values ​​of the weight parameters corresponding to the multiple sample face models.

[0052] Optionally, the second determining module includes:

[0053] A first determining unit is configured to determine first projection position information corresponding to each first reference facial feature point in each face image based on three-dimensional position information corresponding to the first reference facial feature point corresponding to each target facial feature point in each reference facial feature point in the initial three-dimensional face model and device information of each image acquisition device, wherein the reference facial feature points include: facial feature points of various parts of the face, and the same part of the face corresponding to the first reference facial feature point and the target facial feature point that have a corresponding relationship;

[0054] A second determining unit is configured to determine second projection position information corresponding to each target facial feature point in each facial image based on the three-dimensional position information corresponding to the target facial feature point and device information of each image acquisition device;

[0055] The third determination unit is configured to determine the target three-dimensional face model corresponding to the target face based on the second projection position information corresponding to each target face feature point, the first projection position information corresponding to each first reference face feature point and the preset constraint conditions.

[0056] Optionally, the third determining unit is specifically configured to determine a first projection error corresponding to the target face based on the second projection position information corresponding to each target facial feature point, the first projection position information corresponding to each first reference facial feature point, and a preset constraint condition;

[0057] Determine whether the first projection error does not exceed a preset error threshold;

[0058] If it is determined that the first projection error does not exceed a preset error threshold, determining the current initial three-dimensional face model as a target three-dimensional face model corresponding to the target face;

[0059] If it is determined that the first projection error exceeds a preset error threshold, adjusting current values ​​of weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain a current initial three-dimensional face model;

[0060] Determine new first projection position information corresponding to each first reference facial feature point in each facial image based on new three-dimensional position information corresponding to the first reference facial feature point in the current initial three-dimensional facial model and device information of each image acquisition device;

[0061] Based on the second projection position information corresponding to each target facial feature point, the new first projection position information corresponding to each first reference facial feature point and the preset constraints, determine the first projection error corresponding to the target face; return to the step of determining whether the first projection error does not exceed the preset error threshold.

[0062] Optionally, the second determination module is specifically configured to determine the three-dimensional position information corresponding to each second reference facial feature point corresponding to the target facial feature point based on the current values ​​of the multiple sample facial models and their corresponding weight parameters in the initial three-dimensional facial model, wherein the second reference facial feature points and the target facial feature points that have a corresponding relationship correspond to the same part of the face;

[0063] Determine a first distance error corresponding to the target face based on the three-dimensional position information corresponding to each target facial feature point and the three-dimensional position information corresponding to each corresponding second reference facial feature point and preset constraints;

[0064] Determine whether a first distance error corresponding to the target face does not exceed a first distance error threshold;

[0065] If it is determined that the first distance error does not exceed a first distance error threshold, determining the current initial three-dimensional face model as a target three-dimensional face model corresponding to the target face;

[0066] If it is determined that the first distance error exceeds a first distance error threshold, adjusting current values ​​of weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain a current initial three-dimensional face model;

[0067] Based on the adjusted values ​​of the multiple sample face models and their corresponding weight parameters in the current initial three-dimensional face model, determining new three-dimensional position information corresponding to each second reference face feature point;

[0068] Based on the three-dimensional position information corresponding to each target facial feature point and the new three-dimensional position information corresponding to each corresponding second reference facial feature point and the preset constraints, determine the first distance error corresponding to the target face; return to determine whether the first distance error corresponding to the target face does not exceed the first distance error threshold.

[0069] Optionally, the preset constraint condition includes a preset regularization constraint condition and a preset hyperparameter;

[0070] The device also includes: a second obtaining module, which is configured to obtain a preset hyperparameter in the preset constraint before determining the target 3D face model corresponding to the target face based on the initial 3D face model, the preset constraint and the 3D position information corresponding to the target face feature point, wherein the second obtaining module is specifically configured as follows:

[0071] Obtaining real three-dimensional position information corresponding to the debugging face feature points of multiple debugging faces, wherein the debugging face feature points of each debugging face include facial feature points of various parts of the face;

[0072] Obtaining each initial constraint condition, wherein different initial constraints include the same preset regularization constraint condition and different initial hyperparameters;

[0073] For each initial constraint condition, perform the following steps AH to determine a preset constraint condition from the initial constraint condition, and determine a preset hyperparameter in the preset constraint condition;

[0074] Step A: for each debugging face, based on the three-dimensional position information corresponding to the third reference facial feature point at the designated position in the initial three-dimensional facial model, the three-dimensional position information corresponding to the designated debugging facial feature point at the designated position in the debugging facial feature points of the debugging face, and the preset regular constraint condition and initial hyperparameters included in the initial constraint condition, determine the second distance error corresponding to the debugging face, wherein the designated position is the same position as the position of the unobstructed part of the target face;

[0075] Step B: for each debugging face, determining whether the second distance error corresponding to the debugging face does not exceed the second distance error threshold;

[0076] Step C: if it is determined that the second distance error corresponding to the debugging face exceeds the second distance error threshold, adjusting the current values ​​of the weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain the current initial three-dimensional face model;

[0077] Step D: determining new three-dimensional position information corresponding to each third reference facial feature point based on current values ​​of weight parameters corresponding to the multiple sample facial models in the current initial three-dimensional facial model;

[0078] Step E: determining a second distance error corresponding to the debugging face based on the three-dimensional position information corresponding to the designated debugging face feature point at the designated position among the debugging face feature points of the debugging face, the new three-dimensional position information corresponding to the third reference face feature point, and the preset regular constraint condition and initial hyperparameter included in the initial constraint condition; returning to step B;

[0079] Step F: if it is determined that the second distance error corresponding to the debugging face does not exceed the second distance error threshold, the current initial three-dimensional face model is determined as the debugging three-dimensional face model corresponding to the debugging face;

[0080] Step G: determining the debugging three-dimensional position information corresponding to the debugging face feature points at other positions outside the specified position in the debugging face based on the current values ​​of the multiple sample face models and their corresponding weight parameters in the debugging three-dimensional face model corresponding to the debugging face;

[0081] Step H: determining the three-dimensional position error of the debugging face corresponding to the initial constraint condition based on the debugging three-dimensional position information corresponding to the debugging face feature points at other positions outside the specified position in the debugging face and the real three-dimensional position information;

[0082] Based on the three-dimensional position error of each debugging face corresponding to each initial constraint condition, a preset constraint condition is determined from the initial constraint condition, and its initial hyperparameter is determined as the preset hyperparameter, wherein the initial constraint condition with the smallest sum of the three-dimensional position errors of the corresponding debugging faces is determined as the preset constraint condition; or, the initial constraint condition with the smallest number of three-dimensional position errors of the corresponding debugging faces is determined as the preset constraint condition.

[0083] Optionally, the target face is a face wearing a mask, and the unobstructed portion is a portion of the target face that is not obstructed by the mask.

[0084] From the above content, it can be seen that a face recognition method and device provided by an embodiment of the present invention obtains face images collected by multiple image acquisition devices in the same acquisition cycle, wherein the multiple image acquisition devices shoot a target face from different shooting angles, wherein part of the target face is blocked; from the multiple face images, the face feature points of the unblocked part of the target face are detected; based on the face feature points detected in each face image and the device information of each image acquisition device, the three-dimensional position information corresponding to the target face feature points in the target face is determined; based on the initial three-dimensional face model, preset constraints and the three-dimensional position information corresponding to the target face feature points, a target three-dimensional face model corresponding to the target face is determined, wherein the target three-dimensional face model includes the three-dimensional position information corresponding to the face feature points of the blocked part of the target face, and the initial three-dimensional face model includes: multiple sample face models and their corresponding weight parameters, and the preset constraints are used to constrain the values ​​of the weight parameters corresponding to the multiple sample face models.

[0085] By applying the embodiment of the present invention, when part of the target face in the face image is blocked, the three-dimensional position information corresponding to the target face feature points of the unblocked part of the target face can be first constructed, and then the target three-dimensional face model corresponding to the target face can be determined by using the initial three-dimensional face model including multiple sample face models and their corresponding weight parameters, the preset constraint conditions for constraining the values ​​of the weight parameters corresponding to the multiple sample face models and the above three-dimensional position information. The initial three-dimensional face model contains the spatial position information corresponding to the face feature points of the same part as the unblocked part of the target face, and the spatial position information corresponding to the face feature points of the same part as the blocked part of the target face. By adjusting the spatial position of the feature points of the same part as the target face feature points in the initial three-dimensional face model to overlap with the three-dimensional position information corresponding to the target face feature points as much as possible, the spatial position of the feature points of the same part as the blocked part of the target face in the initial three-dimensional face model can be adjusted to obtain the target three-dimensional face model corresponding to the target face. The spatial position information of the feature points in the target three-dimensional face model at the same part as the occluded part of the target face is used as the three-dimensional position information corresponding to the facial feature points at the occluded part of the target face, so as to achieve complete construction of the target face, and then achieve accurate recognition of the partially occluded face. Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present invention.

[0086] The innovative features of the embodiments of the present invention include:

[0087] 1. When part of the target face is blocked in the face image, the three-dimensional position information corresponding to the facial feature points of the unblocked part of the target face can be first constructed, and then the target three-dimensional face model corresponding to the target face can be determined by using an initial three-dimensional face model including multiple sample face models and their corresponding weight parameters and preset constraints for constraining the values ​​of the weight parameters corresponding to the multiple sample face models. The initial three-dimensional face model contains the spatial position information corresponding to the facial feature points of the same part as the unblocked part of the target face and the spatial position information corresponding to the facial feature points of the same part as the blocked part of the target face. The spatial position of the feature points of the same part as the target facial feature points in the initial three-dimensional face model is adjusted to coincide with the three-dimensional position information corresponding to the target facial feature points as much as possible, and the spatial position of the feature points of the same part as the blocked part of the target face in the initial three-dimensional face model is adjusted to obtain the target three-dimensional face model corresponding to the target face. The spatial position information of the feature points in the target three-dimensional face model that are the same as the occluded part of the target face is used as the three-dimensional position information corresponding to the facial feature points in the occluded part of the target face, so as to achieve complete construction of the target face and further accurate recognition of the partially occluded face.

[0088] 2. Determine a first projection error corresponding to the target face by determining the first projection position information corresponding to the first reference facial feature point in the facial image and the second projection position information corresponding to the target facial feature point determined based on the three-dimensional position information corresponding to the target facial feature point, and then determine whether to adjust the spatial position information of the first reference facial feature point based on a comparison between the first projection error and a preset error threshold, wherein adjusting the spatial position information of the first reference facial feature point is achieved by adjusting the values ​​of weight parameters corresponding to multiple sample facial models in the initial three-dimensional face model. When the first projection error does not exceed the preset error threshold, it is considered that the spatial position of the first reference facial feature point is close to the spatial position represented by the three-dimensional position information corresponding to the corresponding target facial feature point, and the current initial three-dimensional facial model is determined to be the target three-dimensional facial model corresponding to the target face; if the first projection error exceeds the preset error threshold, it is considered that the spatial position of the first reference facial feature point is significantly different from the spatial position represented by the three-dimensional position information corresponding to the corresponding target facial feature point, then the spatial position information of the first reference facial feature point, that is, the value of the weight parameter corresponding to the sample face model, is adjusted, and the new first projection error is re-determined until the first projection error does not exceed the preset error threshold, and the target three-dimensional facial model is determined, thereby achieving accurate determination of the target three-dimensional facial model.

[0089] 3. After constructing the three-dimensional position information corresponding to the target facial feature point, directly calculate the spatial distance between the spatial point represented by the three-dimensional position information corresponding to the target facial feature point and the spatial point represented by the three-dimensional position information corresponding to the second reference facial feature point corresponding to the target facial feature point, so as to determine the first distance error corresponding to the target face. If the first distance error does not exceed the first distance error threshold, it is considered that the positions of the two corresponding spatial points are close, and the current initial three-dimensional face model can be determined as the target three-dimensional face model corresponding to the target face; if the first distance error exceeds the first distance error threshold, it is considered that the positions of the two corresponding spatial points are far apart, and the three-dimensional position information corresponding to the second reference facial feature point is adjusted, that is, the current values ​​of the weight parameters corresponding to the multiple sample facial models are adjusted, and then the new three-dimensional position information corresponding to the second reference facial feature point is re-determined, and then the new first distance error is calculated, until the calculated new first distance error does not exceed the first distance error threshold, the target three-dimensional face model is determined, and the target three-dimensional face model is accurately determined.

[0090] 4. For each initial constraint condition, a debugging 3D face model corresponding to the debugging face is constructed using the 3D position information corresponding to the debugging face feature points at the specified positions in the debugging face, the 3D position information corresponding to the third reference face feature points at the specified positions in the initial 3D face model, and the preset regular constraints and initial hyperparameters in the initial constraint conditions; then, based on the debugging 3D position information corresponding to the debugging face feature points at other positions other than the specified positions in the debugging face, and the real 3D position information, the 3D position error of the debugging face corresponding to the initial constraint condition is determined; then, the initial constraint condition with the smallest sum of the 3D position errors of the corresponding debugging face is determined as the preset constraint condition, or the initial constraint condition with the smallest number of 3D position errors of the corresponding debugging face is determined as the preset constraint condition, so as to obtain initial constraints that are more in line with the actual face and provide a basis for ensuring the accuracy of subsequent face recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work.

[0092] Figure 1 A schematic diagram of a flow chart of a face recognition method provided by an embodiment of the present invention;

[0093] Figure 2Another schematic diagram of a face recognition method according to an embodiment of the present invention;

[0094] Figure 3 A schematic diagram of the structure of a face recognition device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0095] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0096] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0097] The present invention provides a face recognition method and device to realize accurate recognition of partially obscured faces. The embodiments of the present invention are described in detail below.

[0098] Figure 1 A flow chart of a face recognition method provided by an embodiment of the present invention. The method may include the following steps:

[0099] S101: Obtaining facial images captured by multiple image acquisition devices in the same acquisition cycle.

[0100] Among them, multiple image acquisition devices shoot the target face from different shooting angles, and some parts of the target face are blocked.

[0101] The face recognition method provided by the embodiment of the present invention can be applied to any type of electronic device with computing capability, and the electronic device can be a server or a terminal.

[0102] The electronic device is connected to a plurality of image acquisition devices, and can obtain images acquired by the plurality of image acquisition devices. In one implementation, the plurality of image acquisition devices can be arranged in the vehicle compartment, and can shoot the face of the driver in the vehicle at multiple angles. In one case, the faces of other passengers in the vehicle can also be photographed. In another implementation, the plurality of image acquisition devices can be arranged in any indoor or outdoor space, and the plurality of image acquisition devices can shoot the faces of people indoors or outdoors at multiple angles. This is all possible, and different image acquisition devices may have different shooting angles for faces.

[0103] In one implementation, when two adjacent image acquisition devices among multiple image acquisition devices take pictures of the target face, their image acquisition areas overlap. It is inevitable that there will be differences in the image acquisition times among multiple image acquisition devices. If the acquisition times corresponding to the images acquired by each image acquisition device are within the preset allowable time error, they can be considered as images acquired within the same acquisition cycle.

[0104] In the embodiment of the present invention, some parts of the target face are blocked, for example, the target face wears a mask, and accordingly, the mouth, nose and chin of the target face are blocked. Another example: the target face wears sunglasses, and accordingly, the eyes of the target face are blocked, etc. The embodiment of the present invention does not limit the specific parts of the target face that are blocked.

[0105] In the case where the target face is wearing a mask, considering that the mask can fit the target face, the detected facial feature points include facial feature points at the contour of the target face. In one case, the detected facial feature points may also include feature points at the tip of the target face's nose.

[0106] S102: Detecting facial feature points of unobstructed parts of a target face from a plurality of facial images.

[0107] In this step, the electronic device can use a pre-trained facial feature point detection model to detect each facial image, and detect facial feature points of unobstructed parts of the target face in each facial image.

[0108] Among them, the pre-trained facial feature point detection model can be: a neural network model trained based on sample facial images labeled with facial feature points, wherein the training process of the pre-trained facial feature point detection model can refer to the training process of the neural network model for detecting facial feature points in the relevant technology, and will not be repeated here.

[0109] S103: Based on the facial feature points detected in each facial image and the device information of each image acquisition device, determine the three-dimensional position information corresponding to the target facial feature points in the target face.

[0110] The target facial feature points are facial feature points whose three-dimensional position information is determined among the detected facial feature points, and may include all or part of the detected facial feature points.

[0111] The device information of each image acquisition device may include device posture information and device internal parameter information. The device internal parameter information of the image acquisition device includes but is not limited to: the length of each pixel point in the horizontal axis direction of the imaging surface of the image acquisition device, the length of each pixel point in the vertical axis direction, the focal length, the position information of the image principal point, and the scaling factor, etc. The image principal point is the intersection of the optical axis of the image acquisition device and the image plane. The device posture information of the image acquisition device may include: the position and posture of the image acquisition device when acquiring the face image.

[0112] In one case, the device posture information of the image acquisition device may be: posture information in the coordinate system of the installation object corresponding to the image acquisition device, wherein the installation object corresponding to the image acquisition system is the object on which the image acquisition system is installed. For example, if multiple image acquisition devices are installed on a vehicle, the vehicle is the installation object corresponding to the image acquisition device, and accordingly, the device posture information of the image acquisition device may be: posture information in the body coordinate system of the vehicle. The image acquisition device has a fixed installation position relative to the vehicle, and accordingly, the posture information of the image acquisition device in the body coordinate system of the vehicle can be considered to be fixed, and the posture conversion relationship between the image acquisition devices remains unchanged.

[0113] In another case, the device posture information of the image acquisition device may be: posture information in the world coordinate system. If multiple image acquisition devices are installed on a moving vehicle, the device posture information of the image acquisition device changes according to the change of the vehicle posture information of the moving vehicle; the relative position relationship between the device posture information of the image acquisition device and the vehicle posture information of the moving vehicle is different, and the relative position relationship between each image acquisition device remains unchanged. In this case, the device posture information of the image acquisition device at each moment can be directly determined based on the vehicle posture information of the moving vehicle at that moment, and the posture conversion relationship between the image acquisition devices remains unchanged.

[0114] In this step, the electronic device can detect facial feature points from each face image and also obtain semantic information corresponding to each detected facial feature point. Accordingly, the electronic device can construct three-dimensional position information corresponding to the facial feature points corresponding to the target semantic information for each target semantic information based on the image position information of the facial feature points in multiple face images corresponding to the target semantic information and the device information corresponding to the face image containing the facial feature points corresponding to the target semantic information. Among them, in the embodiment of the present invention, the same semantic information corresponding to the facial feature points contained in at least two frames of face images is called target semantic information, that is, the target semantic information is: semantic information corresponding to at least two facial feature points belonging to different face images. The facial feature points corresponding to the target semantic information are called target facial feature points.

[0115] The semantic information corresponding to the facial feature points may be: information describing the facial part to which the corresponding facial feature points belong, including but not limited to: information describing the feature points of the eyes, information describing the feature points of the mouth, information describing the feature points of the facial contour, information describing the feature points of the nose, and information describing the feature points of the forehead. For example, the semantic information describing the feature points of the eyes includes but is not limited to: the left inner eye corner point, the left outer eye corner point, the right inner eye corner point, the right outer eye corner point, the Nth upper eyelid point of the left upper eyelid, the tail and brow peak of the left eyebrow, etc.

[0116] S104: Determine a target three-dimensional face model corresponding to the target face based on the initial three-dimensional face model, preset constraints, and three-dimensional position information corresponding to feature points of the target face.

[0117] Among them, the target three-dimensional face model includes the three-dimensional position information corresponding to the facial feature points of the occluded parts of the target face, the initial three-dimensional face model includes: multiple sample face models and their corresponding weight parameters, the preset constraints may include preset regular constraints and preset hyperparameters, and the preset constraints are used to constrain the values ​​of the weight parameters corresponding to the multiple sample face models, so as to combine the initial three-dimensional face model to obtain a target three-dimensional face model that is more in line with the actual facial features.

[0118] In this step, the electronic device can determine the initial three-dimensional position information of each facial feature point in the model face corresponding to the initial three-dimensional face model based on the multiple sample face models included in the initial three-dimensional face model and their corresponding weight parameters; based on the three-dimensional position information corresponding to the target facial feature points in the target face, the facial orientation of the target face can be determined; and then based on the facial orientation of the target face, and the conversion relationship between the spatial rectangular coordinate system where the target face is located and the spatial rectangular coordinate system where the model face is located, the initial three-dimensional position information of each facial feature point in the model face is translated and rotated to determine the three-dimensional position information corresponding to each facial feature point in the model face having the same orientation as the facial orientation of the target face and in the spatial rectangular coordinate system where the target face is located, that is, the three-dimensional position information corresponding to the reference facial feature points in the subsequent initial three-dimensional face model.

[0119] Furthermore, based on the three-dimensional position information of each facial feature point in the model face which has the same facial orientation as the target face and in the rectangular coordinate system of the space where the target face is located, the three-dimensional position information corresponding to the target facial feature points, and preset constraints, the target three-dimensional facial model corresponding to the target face is determined.

[0120] The above-mentioned orientation being the same as the facial orientation of the target face may mean that an angle between the orientation and the facial orientation of the target face is within an allowable angle difference range.

[0121] It is understandable that the three-dimensional position information corresponding to the facial feature points of each part of the face can be obtained based on the initial three-dimensional face model. Subsequently, the target three-dimensional face model obtained will also include the three-dimensional position information corresponding to the facial feature points of each part of the face, that is, it can include the three-dimensional position information corresponding to the facial feature points of the unobstructed parts of the target face and the three-dimensional position information corresponding to the facial feature points of the obstructed parts. Accordingly, the three-dimensional position information corresponding to the facial feature points of the obstructed parts of the target face in the face image can be estimated, and then the target face can be recognized based on the estimated three-dimensional position information corresponding to the facial feature points of the obstructed parts of the target face and the relatively accurate three-dimensional position information corresponding to the facial feature points of the unobstructed parts.

[0122] The multiple sample face models and their corresponding weight parameters included in the initial three-dimensional face model can be obtained through 3DMM (3D morphable model), and the multiple sample face models can include multiple sample face shape models and multiple sample face expression models. The preset constraint conditions include preset regular constraint conditions and preset hyperparameters. The preset regular constraint conditions can refer to the regular constraint conditions in the relevant technology. The preset hyperparameters are screened hyperparameters. For the sake of clear layout, the specific screening method of the preset hyperparameters will be introduced later.

[0123] The preset constraint conditions can realize the position constraint of the three-dimensional position information corresponding to the required facial feature points, so that the three-dimensional position information corresponding to the required facial feature points can meet the characteristics of the actual face, and avoid determining the three-dimensional position information corresponding to the facial feature points that do not conform to the facial characteristics of the face.

[0124] By applying the embodiment of the present invention, when part of the target face is blocked in the face image, the three-dimensional position information corresponding to the facial feature points of the unblocked part of the target face can be first constructed, and then the target three-dimensional face model corresponding to the target face can be determined by using the initial three-dimensional face model including multiple sample face models and their corresponding weight parameters, the preset constraint conditions for constraining the values ​​of the weight parameters corresponding to the multiple sample face models and the above three-dimensional position information. The initial three-dimensional face model contains the spatial position information corresponding to the facial feature points of the same part as the unblocked part of the target face, and the spatial position information corresponding to the facial feature points of the same part as the blocked part of the target face. By adjusting the spatial position of the feature points of the same part as the target facial feature points in the initial three-dimensional face model to overlap with the three-dimensional position information corresponding to the target facial feature points as much as possible, the spatial position of the feature points of the same part as the blocked part of the target face in the initial three-dimensional face model can be adjusted to obtain the target three-dimensional face model corresponding to the target face. The spatial position information of the feature points in the target three-dimensional face model that are the same as the occluded part of the target face is used as the three-dimensional position information corresponding to the facial feature points in the occluded part of the target face, so as to achieve complete construction of the target face and further accurate recognition of the partially occluded face.

[0125] In another embodiment of the present invention, Figure 2 As shown, the S104 may include the following steps:

[0126] S201: Determine the first projection position information corresponding to each first reference facial feature point in each facial image based on the three-dimensional position information corresponding to the first reference facial feature point corresponding to each target facial feature point in each reference facial feature point in the initial three-dimensional facial model and the device information of each image acquisition device.

[0127] The reference facial feature points include: facial feature points of various parts of the face, and the first reference facial feature points and the target facial feature points that have a corresponding relationship correspond to the same part of the face.

[0128] S202: Determine second projection position information corresponding to each target facial feature point in each facial image based on the three-dimensional position information corresponding to the target facial feature point and the device information of each image acquisition device.

[0129] S203: Determine a target three-dimensional face model corresponding to the target face based on the second projection position information corresponding to each target facial feature point, the first projection position information corresponding to each first reference facial feature point, and preset constraints.

[0130] In an embodiment of the present invention, based on multiple sample face models in an initial three-dimensional face model and the current values ​​of their corresponding weight parameters, the three-dimensional position information corresponding to the facial feature points of each part of the model face corresponding to the initial three-dimensional face model can be determined. The facial feature points of each part of the model face can include facial feature points of the same part as the occluded part of the target face, and facial feature points of the same part as the unoccluded part of the target face. In order to describe the situation, the facial feature points of each part of the model face corresponding to the initial three-dimensional face model are called: reference facial feature points in the initial three-dimensional face model.

[0131] The electronic device can first determine the reference facial feature points corresponding to each target facial feature point from the reference facial feature points in the initial three-dimensional face model as the first reference facial feature points, wherein there is a one-to-one correspondence between the target facial feature points and the first reference facial feature points, and the target facial feature points with the corresponding relationship and the first reference facial feature points correspond to the same part of the face, for example: both are the outer corner points of the left eye, or both are the outer corner points of the right eye, or both are the inner corner points of the left eye, and so on.

[0132] Subsequently, the electronic device projects the spatial point represented by the three-dimensional position information corresponding to each first reference facial feature point into each facial image, that is, based on the three-dimensional position information corresponding to the first reference facial feature point and the device information of each image acquisition device, determines the first projection position information corresponding to each first reference facial feature point in each facial image. And projects the spatial point represented by the three-dimensional position information corresponding to each target facial feature point into each facial image, that is, based on the three-dimensional position information corresponding to the target facial feature point and the device information of each image acquisition device, determines the second projection position information corresponding to each target facial feature point in each facial image. It can be understood that the three-dimensional position information corresponding to the target facial feature point integrates the information of the facial feature points in at least two frames of facial images, and the spatial point represented by the three-dimensional position information corresponding to the target facial feature point is projected into each facial image, so that the imaging position of the target facial feature point in the facial image with relatively accurate position can be obtained.

[0133] Furthermore, the electronic device adjusts the initial three-dimensional face model based on the second projection position information corresponding to each target facial feature point, the first projection position information corresponding to each first reference facial feature point, and the preset constraints, that is, adjusts the values ​​of weight parameters corresponding to multiple sample facial models in the initial three-dimensional face model, until the second projection position information corresponding to the target facial feature point and the first projection position information corresponding to the first reference facial feature point corresponding to it meet the preset convergence conditions, and the weight parameters corresponding to multiple sample facial models in the initial three-dimensional face model meet the preset constraints, and determines that the current initial three-dimensional face model is the target three-dimensional face model corresponding to the target face.

[0134] The preset convergence condition may include but is not limited to that the distance between the second projection position information corresponding to the target facial feature point and the first projection position information corresponding to the first reference facial feature point corresponding to the target facial feature point does not exceed a preset distance threshold.

[0135] Accordingly, in another embodiment of the present invention, the S203 may include the following steps 011-016:

[0136] 011: Determine a first projection error corresponding to the target face based on the second projection position information corresponding to each target facial feature point, the first projection position information corresponding to each first reference facial feature point, and preset constraints.

[0137] 012: Determine whether the first projection error does not exceed a preset error threshold.

[0138] 013: If it is determined that the first projection error does not exceed the preset error threshold, the current initial three-dimensional face model is determined as the target three-dimensional face model corresponding to the target face.

[0139] 014: If it is determined that the first projection error exceeds a preset error threshold, current values ​​of weight parameters corresponding to multiple sample face models in the initial three-dimensional face model are adjusted to obtain a current initial three-dimensional face model.

[0140] 015: Based on the new three-dimensional position information corresponding to the first reference facial feature points in the current initial three-dimensional facial model and the device information of each image acquisition device, determine the new first projection position information corresponding to each first reference facial feature point in each facial image.

[0141] 016: Based on the second projection position information corresponding to each target facial feature point, the new first projection position information corresponding to each first reference facial feature point and the preset constraints, determine the first projection error corresponding to the target face; return to 012.

[0142] In an embodiment of the present invention, the electronic device calculates the plane distance error between the projection point of the target facial feature point with a corresponding relationship and the projection point of the first reference facial feature point based on the second projection position information corresponding to each target facial feature point and the first projection position information corresponding to each first reference facial feature point; determines the first projection error corresponding to the target face based on the sum of all the calculated plane distance errors, or the average of the sum of all the plane distance errors, and the preset constraint conditions. It is judged whether the first projection error does not exceed the preset error threshold. If it is judged that the first projection error does not exceed the preset error threshold, it is considered that the distance between the projection point of the target facial feature point with a corresponding relationship and the projection point of the first reference facial feature point is not much different, and accordingly, the distance between the three-dimensional position information corresponding to the target facial feature point with a corresponding relationship and the three-dimensional position information corresponding to the first reference facial feature point is not much different, and the current initial three-dimensional face model is directly determined as the target three-dimensional face model corresponding to the target face. Among them, the preset error threshold is a value set according to the actual situation.

[0143] If it is judged that the first projection error exceeds the preset error threshold, it is considered that the distance between the projection point of the corresponding target facial feature point and the projection point of the first reference facial feature point is greatly different. Accordingly, the distance between the three-dimensional position information corresponding to the corresponding target facial feature point and the three-dimensional position information corresponding to the first reference facial feature point is greatly different. It is necessary to adjust the three-dimensional position information corresponding to each first reference facial feature point, that is, to adjust the current values ​​of the weight parameters corresponding to multiple sample facial models in the initial three-dimensional face model accordingly to obtain the current initial three-dimensional face model.

[0144] Based on the adjusted values ​​of multiple sample face models and their corresponding weight parameters in the current initial three-dimensional face model, determine the new three-dimensional position information corresponding to each first reference face feature point, and determine the new first projection position information corresponding to each first reference face feature point in each face image in combination with the device information of each image acquisition device; then, based on the second projection position information corresponding to each target face feature point and the new first projection position information corresponding to each first reference face feature point, calculate the plane distance error between the projection point of the target face feature point with a corresponding relationship and the projection point of the first reference face feature point; based on the sum of all the calculated plane distance errors, or the average of the sum of all the plane distance errors, and the preset constraint conditions, determine the first projection error corresponding to the target face. Return to determine whether the first projection error does not exceed the preset error threshold, and determine the steps to be performed based on the judgment result. Accordingly, until the first projection error does not exceed the preset error threshold, the current initial three-dimensional face model is determined as the target three-dimensional face model corresponding to the target face, so as to accurately determine the target three-dimensional face model.

[0145] In one case, the initial 3D face model can be expressed by the following formula (1):

[0146]

[0147] Among them, S represents the initial 3D face model, S mean Represents the preset 3D average face model, s i and e j Represents multiple sample face models, where s i represents the i-th sample face shape model in the sample face model, e j represents the jth sample facial expression model in the sample face model, α i represents the weight parameter corresponding to the i-th sample face shape model, m represents the total number of sample face shape models, n represents the total number of sample face expression models, β j is the weight parameter corresponding to the jth sample facial expression model. i and β j are the parameters that need to be solved.

[0148] The above process of determining the first projection error corresponding to the target face can be expressed by the following formula (2):

[0149]

[0150] Among them, Loss1(γ) represents the first projection error corresponding to the target face, T truey represents the second projection position information corresponding to the yth target facial feature point in the xth face image, λ represents the preset hyperparameter, represents the expression corresponding to the preset regular constraint condition, T y represents the first projection position information of the first reference facial feature point corresponding to the yth target facial feature point in the xth face image, p represents the total number of target facial feature points, q represents the total number of face images, and γ t Represents the principal component analysis coefficients with preset regular constraints, including α i and β j σ t represents the principal component deviation under the preset regularization constraint. y，x is determined based on the three-dimensional position information corresponding to the first reference facial feature point corresponding to the yth target facial feature point and the device information of the image acquisition device corresponding to the xth facial image, and the three-dimensional position information corresponding to the first reference facial feature point is determined by the above formula (1); truey，xis determined based on the three-dimensional position information corresponding to the yth target facial feature point and the device information of the image acquisition device corresponding to the xth face image. The value range of t is [1, m+n]. When t is [1, m], γ t =α t , that is, γ 1 ~γ m Take α respectively 1 ~α m , α t represents the weight parameter corresponding to the face shape model of the tth sample; when t is [m+1, m+n], γ t =β t-m , that is, γ m+1 ~γ m+n Take β 1 ~β n , β t-m Represents the weight parameter corresponding to the tmth sample facial expression model.

[0151] The above formula (2) represents the process of determining the first projection error corresponding to the target face based on the calculated sum of all plane distance errors and the preset constraint conditions.

[0152] In another embodiment of the present invention, the S104 may include the following steps 021-027:

[0153] 021: Based on the current values ​​of multiple sample face models and their corresponding weight parameters in the initial three-dimensional face model, preset regularization constraints and preset hyperparameters, determine the three-dimensional position information corresponding to each second reference face feature point corresponding to the target face feature point.

[0154] The second reference facial feature points and the target facial feature points that have a corresponding relationship correspond to the same part of the face.

[0155] 022: Determine a first distance error corresponding to the target face based on the three-dimensional position information corresponding to each target facial feature point and the three-dimensional position information corresponding to each corresponding second reference facial feature point and preset constraints.

[0156] 023: Determine whether the first distance error corresponding to the target face does not exceed the first distance error threshold.

[0157] 024: If it is determined that the first distance error does not exceed the first distance error threshold, the current initial three-dimensional face model is determined as the target three-dimensional face model corresponding to the target face.

[0158] 025: If it is determined that the first distance error exceeds the first distance error threshold, adjust current values ​​of weight parameters corresponding to multiple sample face models in the initial three-dimensional face model to obtain a current initial three-dimensional face model.

[0159] 026: Based on the adjusted values ​​of the multiple sample face models and their corresponding weight parameters in the current initial three-dimensional face model, determine the new three-dimensional position information corresponding to each second reference face feature point.

[0160] 027: Based on the three-dimensional position information corresponding to each target facial feature point and the new three-dimensional position information corresponding to each corresponding second reference facial feature point and the preset constraint conditions, determine the first distance error corresponding to the target face; return to 023.

[0161] In the embodiment of the present invention, the target three-dimensional face model corresponding to the target face can also be determined by directly calculating the spatial distance between the spatial point represented by the three-dimensional position information corresponding to the target face feature point and the spatial point represented by the three-dimensional position information corresponding to the corresponding reference face feature point. Accordingly, the electronic device determines the three-dimensional position information corresponding to the reference face feature points of each part in the average face corresponding to the initial three-dimensional face model based on the current values ​​of the multiple sample face models and their corresponding weight parameters in the initial three-dimensional face model.

[0162] A reference facial feature point corresponding to each target facial feature point is determined from the reference facial feature points as a second reference facial feature point corresponding to the target facial feature point, wherein there is a one-to-one correspondence between the target facial feature point and the second reference facial feature point, and the target facial feature point with a corresponding relationship and the second reference facial feature point correspond to the same part of the face, for example: both are the outer corner points of the left eye, or both are the outer corner points of the right eye, or both are the inner corner points of the left eye, etc.

[0163] Based on the three-dimensional position information corresponding to each target facial feature point and the three-dimensional position information corresponding to each corresponding second reference facial feature point, calculate the spatial distance error between each pair of spatial points corresponding to the target facial feature points and the spatial points corresponding to the second reference facial feature points that have a corresponding relationship; based on the sum of all the calculated spatial distance errors, or the average of the sum of all the spatial distance errors, and the preset constraints, determine the first distance error corresponding to the target face. Determine whether the first distance error corresponding to the target face does not exceed the first distance error threshold. Among them, the spatial point corresponding to the target facial feature point is the spatial point represented by its corresponding three-dimensional position information, and the spatial point corresponding to the second reference facial feature point is the spatial point represented by its corresponding three-dimensional position information.

[0164] If it is judged that the first distance error does not exceed the first distance error threshold, it is considered that the distance between the spatial point corresponding to the target facial feature point with a corresponding relationship and the spatial point corresponding to the second reference facial feature point is not much different, and the current initial three-dimensional face model is directly determined as the target three-dimensional face model corresponding to the target face.

[0165] If it is judged that the first distance error exceeds the first distance error threshold, it is considered that the distance between the spatial point corresponding to the corresponding target facial feature point and the spatial point corresponding to the second reference facial feature point is greatly different, then it is necessary to adjust the position of the spatial point corresponding to the second reference facial feature point, that is, adjust the three-dimensional position information corresponding to the second reference facial feature point, and specifically adjust the current values ​​of the weight parameters corresponding to multiple sample facial models in the initial three-dimensional face model to obtain the current initial three-dimensional face model.

[0166] Then, based on the adjusted values ​​of multiple sample face models and their corresponding weight parameters in the current initial three-dimensional face model, the new three-dimensional position information corresponding to each second reference face feature point is determined. The spatial distance error between each pair of space points corresponding to the target face feature points and the space points corresponding to the second reference face feature points that have a corresponding relationship is calculated again; based on the sum of all the calculated spatial distance errors, or the average of the sum of all the spatial distance errors, and the preset constraints, the first distance error corresponding to the target face is determined, and it is determined whether the first distance error corresponding to the target face does not exceed the first distance error threshold, and then the steps to be performed are determined based on the judgment result. Accordingly, until it is determined that the first distance error corresponding to the target face does not exceed the first distance error threshold, the current initial three-dimensional face model is determined as the target three-dimensional face model corresponding to the target face, so as to accurately determine the target three-dimensional face model.

[0167] In one case, the initial 3D face model can be expressed by the above formula (1).

[0168] The above process of determining the first distance error corresponding to the target face can be expressed by the following formula (3):

[0169]

[0170] Among them, Loss2(v) represents the first distance error corresponding to the target face, S truez represents the three-dimensional position information corresponding to the z-th target facial feature point, λ represents the preset hyperparameter, represents the expression corresponding to the preset regular constraint condition, S z represents the 3D position information of the first reference facial feature point corresponding to the zth target facial feature point in the initial 3D face model, k represents the total number of target facial feature points, γ t Represents the principal component analysis coefficients with preset regular constraints, including α i and β j σ tRepresents the principal component deviation of the preset regular constraint condition. The value range of t is [1, m+n]. When t is [1, m], γ t =α t , that is, γ 1 ~γ m Take α respectively 1 ~α m , α t represents the weight parameter corresponding to the face shape model of the tth sample; when t is [m+1, m+n], γ t =β t-m , that is, γ m+1 ~γ m+n Take β 1 ~β n , β t-m Represents the weight parameter corresponding to the tmth sample facial expression model.

[0171] The above formula (3) represents the process of determining the first distance error corresponding to the target face based on the calculated sum of all spatial distance errors and the preset constraint conditions.

[0172] In order to ensure that the determined target face model is more consistent with normal face characteristics, and to ensure the accuracy of the recognition results of the face recognition process provided by the embodiment of the present invention, when determining the target three-dimensional face model based on the initial three-dimensional face model, it is necessary to constrain the weight parameters corresponding to multiple sample face models in the three-dimensional face model through appropriate preset constraints to ensure the balance between the factors that affect the construction of the target face. In another embodiment of the present invention, the preset constraints include preset regular constraints and preset hyperparameters, wherein the accurate selection of the preset hyperparameters can improve the accuracy of the determined target three-dimensional face model to a certain extent; before S104, the method may also include step 03:

[0173] 03: Obtaining preset hyperparameters in the preset constraint conditions, wherein step 03 may include the following steps:

[0174] 031: Obtaining real three-dimensional position information corresponding to debugging facial feature points of multiple debugging faces, wherein the debugging facial feature points of each debugging face include facial feature points of various parts of the face;

[0175] 032: Get each initial constraint.

[0176] Among them, different initial constraints include the same preset regularization constraints but different initial hyperparameters.

[0177] 033: For each initial constraint condition, execute the following steps AH to determine the preset constraint condition from the initial constraint condition, and determine the preset hyperparameter in the preset constraint condition.

[0178] Step A: For each debugging face, based on the three-dimensional position information corresponding to the third reference facial feature point at the specified position in the initial three-dimensional face model, the three-dimensional position information corresponding to the specified debugging facial feature point at the specified position in the debugging facial feature points of the debugging face, and the preset regular constraint conditions and initial hyperparameters included in the initial constraint conditions, determine the second distance error corresponding to the debugging face.

[0179] The designated position is the same as the position of the unobstructed part of the target face.

[0180] Step B: for each debugging face, determining whether the second distance error corresponding to the debugging face does not exceed the second distance error threshold.

[0181] Step C: If it is determined that the second distance error corresponding to the debugging face exceeds the second distance error threshold, adjust the current values ​​of the weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain the current initial three-dimensional face model.

[0182] Step D: Based on the current values ​​of the weight parameters corresponding to the multiple sample face models in the current initial three-dimensional face model, determine the new three-dimensional position information corresponding to each third reference face feature point.

[0183] Step E: Determine the second distance error corresponding to the debugging face based on the three-dimensional position information corresponding to the designated debugging facial feature point at the designated position among the debugging facial feature points of the debugging face, the new three-dimensional position information corresponding to the third reference facial feature point, and the preset regular constraint condition and initial hyperparameter included in the initial constraint condition; return to step B.

[0184] Step F: If it is determined that the second distance error corresponding to the debugging face does not exceed the second distance error threshold, the current initial three-dimensional face model is determined as the debugging three-dimensional face model corresponding to the debugging face.

[0185] Step G: Based on the current values ​​of multiple sample face models and their corresponding weight parameters in the debugging 3D face model corresponding to the debugging face, determine the debugging 3D position information corresponding to the debugging face feature points at other positions outside the specified position in the debugging face.

[0186] Step H: Based on the debugging 3D position information corresponding to the debugging facial feature points at other positions outside the specified position in the debugging face and the real 3D position information, determine the 3D position error of the debugging face corresponding to the initial constraint condition.

[0187] 034: Based on the three-dimensional position error of each debugging face corresponding to each initial constraint condition, a preset constraint condition is determined from the initial constraint conditions, and its initial hyperparameter is determined as the preset hyperparameter.

[0188] Among them, the initial constraint condition with the smallest sum of the three-dimensional position errors of the corresponding debugging face is determined as the preset constraint condition; or the initial constraint condition with the largest number of the smallest three-dimensional position errors of the corresponding debugging face is determined as the preset constraint condition.

[0189] In the embodiment of the present invention, the electronic device locally or in a connected storage device pre-stores a plurality of initial constraints, wherein different initial constraints include the same preset regular constraints but different initial hyperparameters. Before executing S104, the electronic device may first determine the preset constraints from the plurality of initial constraints, that is, determine the preset hyperparameters in the preset constraints.

[0190] Accordingly, the electronic device can obtain the real three-dimensional position information corresponding to the debugging face feature points of multiple debugging faces, and the debugging face feature points of each debugging face include the facial feature points of each part of the face. Then, for each initial constraint condition, the three-dimensional position error corresponding to the initial constraint condition is determined, that is, the three-dimensional position error corresponding to each debugging face corresponding to the initial constraint condition, and then based on the three-dimensional position error corresponding to each initial constraint condition, the preset constraint condition is determined from the initial constraint condition, and its initial hyperparameter is determined as the preset hyperparameter.

[0191] Specifically, for each debugging face, based on the current values ​​of multiple sample face models and their corresponding weight parameters in the initial three-dimensional face model, the three-dimensional position information corresponding to the facial feature points of each part of the model face corresponding to the initial three-dimensional face model is determined; based on the three-dimensional position information corresponding to the third reference facial feature points at the specified positions in the initial three-dimensional face model, and the three-dimensional position information corresponding to the specified debugging facial feature points at the specified positions in the debugging facial feature points of the debugging face, the distance error between each pair of spatial points corresponding to the third reference facial feature points at the same position and the spatial points corresponding to the specified debugging facial feature points is calculated, and then, based on the sum of all distance errors, or the average of the sum of all distance errors, and the preset regular constraint conditions and initial hyperparameters included in the initial constraint conditions, the second distance error corresponding to the debugging face is determined.

[0192] For each debugging face, determine whether the second distance error corresponding to the debugging face does not exceed the second distance error threshold. If it is determined that the second distance error corresponding to the debugging face exceeds the second distance error threshold, it is considered that the distance between the spatial point corresponding to the third reference facial feature point at the same position and the spatial point corresponding to the specified debugging facial feature point is relatively different, and it is necessary to adjust the position of the spatial point corresponding to the third reference facial feature point, that is, adjust the current values ​​of the weight parameters corresponding to the multiple sample facial models in the initial three-dimensional face model to obtain the current initial three-dimensional face model; then, based on the multiple sample facial models in the current initial three-dimensional face model and the adjusted values ​​of the corresponding weight parameters, determine the new three-dimensional position corresponding to each third reference facial feature point. The setting information is obtained, and the second distance error corresponding to the debugging face is determined again based on the new three-dimensional position information corresponding to the designated debugging facial feature point at the designated position in the debugging facial feature points of the debugging face, the three-dimensional position information corresponding to the third reference facial feature point, and the preset regular constraint conditions and initial hyperparameters included in the initial constraint conditions, and it is judged whether the second distance error corresponding to the debugging face does not exceed the second distance error threshold. Until it is judged that the second distance error corresponding to the debugging face does not exceed the second distance error threshold, it is considered that the distance between the spatial point corresponding to the third reference facial feature point at the same position and the spatial point corresponding to the designated debugging facial feature point is not much different, and the current initial three-dimensional face model is determined as the debugging three-dimensional face model corresponding to the debugging face.

[0193] Then, based on the current values ​​of multiple sample face models and their corresponding weight parameters in the debugging three-dimensional face model corresponding to the debugging face, the debugging three-dimensional position information corresponding to the debugging face feature points at other positions other than the specified position in the debugging face is determined. For each debugging face feature point at other positions other than the specified position in the debugging face, the distance error between the debugging three-dimensional position information corresponding to the debugging face feature point and the real three-dimensional position information is calculated as the distance error corresponding to the debugging face feature point, and the sum of the distance errors corresponding to all the debugging face feature points at other positions other than the specified position in the debugging face, or the average of the sum of the distance errors corresponding to all the debugging face feature points at other positions other than the specified position in the debugging face, is used as the three-dimensional position error of the debugging face corresponding to the initial constraint condition, so as to determine the three-dimensional position errors of all the debugging faces corresponding to the initial constraint condition, and then determine the three-dimensional position errors of all the debugging faces corresponding to each first three-dimensional face model.

[0194] For example, the initial constraints include 3 and the debugging faces include 20. Then, for each initial constraint, the three-dimensional position errors corresponding to the 20 debugging faces can be determined, that is, each initial constraint corresponds to 20 three-dimensional position errors.

[0195] Subsequently, the electronic device determines the preset constraint condition from the initial constraint condition based on the three-dimensional position error of each debugging face corresponding to each initial constraint condition, and determines its initial hyperparameter as the preset hyperparameter. In one implementation, the initial constraint condition with the smallest sum of the three-dimensional position errors of the corresponding debugging faces is determined as the preset constraint condition. Specifically, for each initial constraint condition, the sum of the three-dimensional position errors of each debugging face corresponding to the initial constraint condition is used as the sum of the three-dimensional position errors corresponding to the initial constraint condition; the initial constraint condition with the smallest sum of the corresponding three-dimensional position errors is determined as the preset constraint condition, and accordingly, the initial hyperparameter of the initial constraint condition with the smallest sum of the corresponding three-dimensional position errors is determined as the preset hyperparameter.

[0196] Another implementation is to determine the initial constraint condition with the smallest number of three-dimensional position errors of the corresponding debugging face as the preset constraint condition. Specifically, for each debugging face, compare the three-dimensional position errors of all initial constraints corresponding to the debugging face, determine the initial constraint condition with the smallest three-dimensional position error of the debugging face from all initial constraints, and add one to the minimum value count corresponding to the initial constraint condition with the smallest three-dimensional position error of the debugging face; count the values ​​of the minimum value count corresponding to each initial constraint condition, determine the initial constraint condition with the smallest value of the corresponding minimum value count as the preset constraint condition, and accordingly, determine its initial hyperparameter as the preset hyperparameter.

[0197] In one implementation, the real three-dimensional position information corresponding to the debugging face feature points of multiple debugging faces may be collected by a depth image acquisition device.

[0198] In another implementation, the real three-dimensional position information corresponding to the debugging face feature points of multiple debugging faces can be determined by the images containing the debugging faces collected by multiple image acquisition devices in the same acquisition cycle and the device information of the image acquisition devices corresponding to the images. Among them, each debugging face feature point can be first detected from the image containing the debugging face, and then the real three-dimensional position information corresponding to each debugging face feature point can be determined by using the position information of the debugging face feature point in each image containing the debugging face and the device information of the image acquisition device corresponding to each image.

[0199] Accordingly, in one implementation, when the real three-dimensional position information corresponding to the debugging face feature points of multiple debugging faces is determined by images containing the debugging faces collected by multiple image acquisition devices in the same acquisition cycle and the device information of the image acquisition devices corresponding to the images, the electronic device, in the process of determining the debugging three-dimensional face model corresponding to each debugging face, may, for each debugging face, use the real three-dimensional position information corresponding to the debugging face feature points at other positions of the debugging face except the specified position, and the device information of the image acquisition device that acquires the image containing the debugging face, to determine first debugging position information of the spatial point represented by the real three-dimensional position information corresponding to the debugging face feature points at each other position at the projection point of the image containing the debugging face; use the debugging three-dimensional position information corresponding to the debugging face feature points at other positions except the specified position of the debugging face, and the device information of the image acquisition device that acquires the image containing the debugging face, to determine second debugging position information of the spatial point represented by the debugging three-dimensional position information corresponding to the debugging face feature points at other positions at the projection point of the image containing the debugging face, and then determine the debugging three-dimensional face model corresponding to the debugging face based on the first debugging position information and the corresponding second debugging position information. Then execute the subsequent process.

[0200] Corresponding to the above method embodiment, the embodiment of the present invention provides a face recognition device, such as Figure 3 As shown, the device comprises:

[0201] The first acquisition module 310 is configured to obtain facial images acquired by multiple image acquisition devices in the same acquisition cycle, wherein the multiple image acquisition devices capture the target face from different shooting angles, and part of the target face is blocked;

[0202] The detection module 320 is configured to detect facial feature points of unobstructed parts of the target face from the multiple facial images;

[0203] The first determination module 330 is configured to determine the three-dimensional position information corresponding to the target facial feature points in the target face based on the facial feature points detected in each facial image and the device information of each image acquisition device;

[0204] The second determination module 340 is configured to determine the target three-dimensional face model corresponding to the target face based on the initial three-dimensional face model, preset constraints and the three-dimensional position information corresponding to the target face feature points, wherein the target three-dimensional face model includes the three-dimensional position information corresponding to the facial feature points of the occluded parts of the target face, the initial three-dimensional face model includes: multiple sample face models and their corresponding weight parameters, and the preset constraints are used to constrain the values ​​of the weight parameters corresponding to the multiple sample face models.

[0205] By using the embodiment of the present invention, when part of the target face is blocked in the face image, the three-dimensional position information corresponding to the facial feature points of the unblocked part of the target face can be first constructed, and then, the target three-dimensional face model corresponding to the target face can be determined by using the initial three-dimensional face model including multiple sample face models and their corresponding weight parameters, the preset constraint conditions for constraining the values ​​of the weight parameters corresponding to the multiple sample face models and the above three-dimensional position information. The initial three-dimensional face model contains the spatial position information corresponding to the facial feature points of the same part as the unblocked part of the target face, and the spatial position information corresponding to the facial feature points of the same part as the blocked part of the target face. By adjusting the spatial position of the feature points of the same part as the target facial feature points in the initial three-dimensional face model to overlap with the three-dimensional position information corresponding to the target facial feature points as much as possible, and at the same time, the spatial position of the feature points of the same part as the blocked part of the target face in the initial three-dimensional face model is adjusted, and the target three-dimensional face model corresponding to the target face is obtained. The spatial position information of the feature points in the target three-dimensional face model that are the same as the occluded part of the target face is used as the three-dimensional position information corresponding to the facial feature points in the occluded part of the target face, so as to achieve complete construction of the target face and further accurate recognition of the partially occluded face.

[0206] In another embodiment of the present invention, the second determining module 340 includes:

[0207] A first determination unit (not shown in the figure) is configured to determine the first projection position information corresponding to each first reference facial feature point in each face image based on the three-dimensional position information corresponding to the first reference facial feature point corresponding to each target facial feature point in each reference facial feature point in the initial three-dimensional face model and the device information of each image acquisition device, wherein the reference facial feature points include: facial feature points of various parts of the face, the first reference facial feature points and the target facial feature points that have a corresponding relationship correspond to the same part of the face, and the three-dimensional position information corresponding to each reference facial feature point in the initial three-dimensional face model is determined based on the current values ​​of the multiple sample face models and their corresponding weight parameters, the preset regularization constraints, and the preset hyperparameters;

[0208] A second determining unit (not shown in the figure) is configured to determine second projection position information corresponding to each target facial feature point in each facial image based on the three-dimensional position information corresponding to the target facial feature point and the device information of each image acquisition device;

[0209] The third determination unit (not shown in the figure) is configured to determine the target three-dimensional face model corresponding to the target face based on the second projection position information corresponding to each target facial feature point, the first projection position information corresponding to each first reference facial feature point and preset constraints.

[0210] In another embodiment of the present invention, the third determining unit is specifically configured to determine the first projection error corresponding to the target face based on the second projection position information corresponding to each target facial feature point, the first projection position information corresponding to each first reference facial feature point, and a preset constraint condition;

[0211] Determine whether the first projection error does not exceed a preset error threshold;

[0212] If it is determined that the first projection error does not exceed a preset error threshold, determining the current initial three-dimensional face model as a target three-dimensional face model corresponding to the target face;

[0213] If it is determined that the first projection error exceeds a preset error threshold, adjusting current values ​​of weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain a current initial three-dimensional face model;

[0214] Determine new first projection position information corresponding to each first reference facial feature point in each facial image based on new three-dimensional position information corresponding to the first reference facial feature point in the current initial three-dimensional facial model and device information of each image acquisition device;

[0215] Based on the second projection position information corresponding to each target facial feature point, the new first projection position information corresponding to each first reference facial feature point and the preset constraints, determine the first projection error corresponding to the target face; return to the step of determining whether the first projection error does not exceed the preset error threshold.

[0216] In another embodiment of the present invention, the second determination module 340 is specifically configured to determine the three-dimensional position information corresponding to each second reference facial feature point corresponding to the target facial feature point based on the current values ​​of the multiple sample facial models and their corresponding weight parameters in the initial three-dimensional facial model, the preset regularization constraint and the preset hyperparameter, wherein the second reference facial feature points and the target facial feature points that have a corresponding relationship correspond to the same part of the face;

[0217] Determine a first distance error corresponding to the target face based on the three-dimensional position information corresponding to each target facial feature point and the three-dimensional position information corresponding to each corresponding second reference facial feature point and preset constraints;

[0218] Determine whether a first distance error corresponding to the target face does not exceed a first distance error threshold;

[0219] If it is determined that the first distance error does not exceed a first distance error threshold, determining the current initial three-dimensional face model as a target three-dimensional face model corresponding to the target face;

[0220] If it is determined that the first distance error exceeds a first distance error threshold, adjusting current values ​​of weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain a current initial three-dimensional face model;

[0221] Based on the adjusted values ​​of the multiple sample face models and their corresponding weight parameters in the current initial three-dimensional face model, determining new three-dimensional position information corresponding to each second reference face feature point;

[0222] Based on the three-dimensional position information corresponding to each target facial feature point and the new three-dimensional position information corresponding to each corresponding second reference facial feature point and the preset constraints, determine the first distance error corresponding to the target face; return to determine whether the first distance error corresponding to the target face does not exceed the first distance error threshold.

[0223] In another embodiment of the present invention, the preset constraints include preset regularization constraints and preset hyperparameters;

[0224] The device further includes: a second obtaining module (not shown in the figure), configured to obtain a preset hyperparameter in the preset constraint before determining the target three-dimensional face model corresponding to the target face based on the initial three-dimensional face model, the preset constraint and the three-dimensional position information corresponding to the target face feature points, wherein the second obtaining module (not shown in the figure) is specifically configured to obtain real three-dimensional position information corresponding to the debugging face feature points of multiple debugging faces, wherein the debugging face feature points of each debugging face include facial feature points of various parts of the face;

[0225] Obtaining real three-dimensional position information corresponding to the debugging face feature points of multiple debugging faces, wherein the debugging face feature points of each debugging face include facial feature points of various parts of the face;

[0226] Obtaining each initial constraint condition, wherein different initial constraints include the same preset regularization constraint condition and different initial hyperparameters;

[0227] For each initial constraint condition, perform the following steps AH to determine a preset constraint condition from the initial constraint condition, and determine a preset hyperparameter in the preset constraint condition;

[0228] Step A: for each debugging face, based on the three-dimensional position information corresponding to the third reference facial feature point at the designated position in the initial three-dimensional facial model, the three-dimensional position information corresponding to the designated debugging facial feature point at the designated position in the debugging facial feature points of the debugging face, and the preset regular constraint condition and initial hyperparameters included in the initial constraint condition, determine the second distance error corresponding to the debugging face, wherein the designated position is the same position as the position of the unobstructed part of the target face;

[0229] Step B: for each debugging face, determining whether the second distance error corresponding to the debugging face does not exceed the second distance error threshold;

[0230] Step C: if it is determined that the second distance error corresponding to the debugging face exceeds the second distance error threshold, adjusting the current values ​​of the weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain the current initial three-dimensional face model;

[0231] Step D: determining new three-dimensional position information corresponding to each third reference facial feature point based on current values ​​of weight parameters corresponding to the multiple sample facial models in the current initial three-dimensional facial model;

[0232] Step E: determining a second distance error corresponding to the debugging face based on the three-dimensional position information corresponding to the designated debugging face feature point at the designated position among the debugging face feature points of the debugging face, the new three-dimensional position information corresponding to the third reference face feature point, and the preset regular constraint condition and initial hyperparameter included in the initial constraint condition; returning to step B;

[0233] Step F: if it is determined that the second distance error corresponding to the debugging face does not exceed the second distance error threshold, the current initial three-dimensional face model is determined as the debugging three-dimensional face model corresponding to the debugging face;

[0234] Step G: determining the debugging three-dimensional position information corresponding to the debugging face feature points at other positions outside the specified position in the debugging face based on the current values ​​of the multiple sample face models and their corresponding weight parameters in the debugging three-dimensional face model corresponding to the debugging face;

[0235] Step H: determining the three-dimensional position error of the debugging face corresponding to the initial constraint condition based on the debugging three-dimensional position information corresponding to the debugging face feature points at other positions outside the specified position in the debugging face and the real three-dimensional position information;

[0236] Based on the three-dimensional position error of each debugging face corresponding to each initial constraint condition, a preset constraint condition is determined from the initial constraint condition, and its initial hyperparameter is determined as the preset hyperparameter, wherein the initial constraint condition with the smallest sum of the three-dimensional position errors of the corresponding debugging faces is determined as the preset constraint condition; or, the initial constraint condition with the smallest number of three-dimensional position errors of the corresponding debugging faces is determined as the preset constraint condition.

[0237] In another embodiment of the present invention, the target face is a face wearing a mask, and the unobstructed portion is a portion of the target face that is not obstructed by the mask.

[0238] The above-mentioned device and system embodiments correspond to the method embodiments and have the same technical effects as the method embodiments. For specific descriptions, please refer to the method embodiments. The device embodiments are obtained based on the method embodiments. For specific descriptions, please refer to the method embodiment part, which will not be repeated here. A person of ordinary skill in the art can understand that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present invention.

[0239] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0240] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A face recognition method, characterized in that: The method comprises: Obtaining facial images captured by multiple image acquisition devices in the same acquisition cycle, wherein the multiple image acquisition devices capture the target face from different shooting angles, and part of the target face is blocked; Detecting facial feature points of unobstructed parts of the target face from the multiple facial images; Determine the three-dimensional position information corresponding to the target facial feature points in the target face based on the facial feature points detected in each facial image and the device information of each image acquisition device; Determine a target three-dimensional face model corresponding to the target face based on the initial three-dimensional face model, preset constraints and the three-dimensional position information corresponding to the target face feature points, wherein the target three-dimensional face model includes the three-dimensional position information corresponding to the face feature points of the occluded parts of the target face, the initial three-dimensional face model includes: a plurality of sample face models and their corresponding weight parameters, and the preset constraints are used to constrain the values ​​of the weight parameters corresponding to the plurality of sample face models; The step of determining a target three-dimensional face model corresponding to the target face based on the initial three-dimensional face model, preset constraints and the three-dimensional position information corresponding to the target face feature points comprises: Based on the three-dimensional position information corresponding to the first reference facial feature point corresponding to each target facial feature point in each reference facial feature point in the initial three-dimensional facial model and the device information of each image acquisition device, determine the first projection position information corresponding to each first reference facial feature point in each facial image, wherein the reference facial feature points include: facial feature points of various parts of the face, and the same parts of the face corresponding to the first reference facial feature points and the target facial feature points that have a corresponding relationship; Determine second projection position information corresponding to each target facial feature point in each facial image based on the three-dimensional position information corresponding to the target facial feature point and the device information of each image acquisition device; Based on the second projection position information corresponding to each target facial feature point, the first projection position information corresponding to each first reference facial feature point and preset constraints, a target three-dimensional facial model corresponding to the target face is determined.

2. The method according to claim 1, characterized in that The step of determining a target three-dimensional face model corresponding to the target face based on the second projection position information corresponding to each target face feature point, the first projection position information corresponding to each first reference face feature point, and preset constraints includes: Determine a first projection error corresponding to the target face based on the second projection position information corresponding to each target facial feature point, the first projection position information corresponding to each first reference facial feature point, and a preset constraint condition; Determine whether the first projection error does not exceed a preset error threshold; If it is determined that the first projection error does not exceed a preset error threshold, determining the current initial three-dimensional face model as a target three-dimensional face model corresponding to the target face; If it is determined that the first projection error exceeds a preset error threshold, adjusting current values ​​of weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain a current initial three-dimensional face model; Determine new first projection position information corresponding to each first reference facial feature point in each facial image based on new three-dimensional position information corresponding to the first reference facial feature point in the current initial three-dimensional facial model and device information of each image acquisition device; Based on the second projection position information corresponding to each target facial feature point, the new first projection position information corresponding to each first reference facial feature point and the preset constraints, determine the first projection error corresponding to the target face; return to the step of determining whether the first projection error does not exceed the preset error threshold.

3. The method according to claim 1, characterized in that The step of determining a target three-dimensional face model corresponding to the target face based on the initial three-dimensional face model, preset constraints and the three-dimensional position information corresponding to the target face feature points comprises: Based on the current values ​​of the multiple sample face models and their corresponding weight parameters in the initial three-dimensional face model, determine the three-dimensional position information corresponding to each second reference face feature point corresponding to the target face feature point, wherein the second reference face feature point and the target face feature point that have a corresponding relationship correspond to the same part of the face; Determine a first distance error corresponding to the target face based on the three-dimensional position information corresponding to each target facial feature point and the three-dimensional position information corresponding to each corresponding second reference facial feature point and preset constraints; Determine whether a first distance error corresponding to the target face does not exceed a first distance error threshold; If it is determined that the first distance error does not exceed a first distance error threshold, determining the current initial three-dimensional face model as a target three-dimensional face model corresponding to the target face; If it is determined that the first distance error exceeds a first distance error threshold, adjusting current values ​​of weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain a current initial three-dimensional face model; Based on the adjusted values ​​of the multiple sample face models and their corresponding weight parameters in the current initial three-dimensional face model, determining new three-dimensional position information corresponding to each second reference face feature point; Based on the three-dimensional position information corresponding to each target facial feature point and the new three-dimensional position information corresponding to each corresponding second reference facial feature point and the preset constraints, determine the first distance error corresponding to the target face; return to the step of determining whether the first distance error corresponding to the target face does not exceed the first distance error threshold.

4. The method according to any one of claims 1 to 3, characterized in that: The preset constraints include preset regular constraints and preset hyperparameters; Before the step of determining a target 3D face model corresponding to the target face based on the initial 3D face model, the preset constraints and the 3D position information corresponding to the target face feature points, the method further includes: The process of obtaining a preset hyperparameter in a preset constraint condition includes: Obtaining real three-dimensional position information corresponding to the debugging face feature points of multiple debugging faces, wherein the debugging face feature points of each debugging face include facial feature points of various parts of the face; Obtaining each initial constraint condition, wherein different initial constraints include the same preset regularization constraint condition and different initial hyperparameters; For each initial constraint condition, perform the following steps AH to determine a preset constraint condition from the initial constraint condition, and determine a preset hyperparameter in the preset constraint condition; Step A: for each debugging face, based on the three-dimensional position information corresponding to the third reference facial feature point at the designated position in the initial three-dimensional facial model, the three-dimensional position information corresponding to the designated debugging facial feature point at the designated position in the debugging facial feature points of the debugging face, and the preset regular constraint condition and initial hyperparameters included in the initial constraint condition, determine the second distance error corresponding to the debugging face, wherein the designated position is the same position as the position of the unobstructed part of the target face; Step B: for each debugging face, determining whether the second distance error corresponding to the debugging face does not exceed the second distance error threshold; Step C: if it is determined that the second distance error corresponding to the debugging face exceeds the second distance error threshold, adjusting the current values ​​of the weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain the current initial three-dimensional face model; Step D: determining new three-dimensional position information corresponding to each third reference facial feature point based on current values ​​of weight parameters corresponding to the multiple sample facial models in the current initial three-dimensional facial model; Step E: determining a second distance error corresponding to the debugging face based on the three-dimensional position information corresponding to the designated debugging face feature point at the designated position among the debugging face feature points of the debugging face, the new three-dimensional position information corresponding to the third reference face feature point, and the preset regular constraint condition and initial hyperparameter included in the initial constraint condition; returning to step B; Step F: if it is determined that the second distance error corresponding to the debugging face does not exceed the second distance error threshold, the current initial three-dimensional face model is determined as the debugging three-dimensional face model corresponding to the debugging face; Step G: determining the debugging three-dimensional position information corresponding to the debugging face feature points at other positions outside the specified position in the debugging face based on the current values ​​of the multiple sample face models and their corresponding weight parameters in the debugging three-dimensional face model corresponding to the debugging face; Step H: determining the three-dimensional position error of the debugging face corresponding to the initial constraint condition based on the debugging three-dimensional position information corresponding to the debugging face feature points at other positions outside the specified position in the debugging face and the real three-dimensional position information; Based on the three-dimensional position error of each debugging face corresponding to each initial constraint condition, a preset constraint condition is determined from the initial constraint condition, and its initial hyperparameter is determined as the preset hyperparameter, wherein the initial constraint condition with the smallest sum of the three-dimensional position errors of the corresponding debugging faces is determined as the preset constraint condition; or, the initial constraint condition with the smallest number of three-dimensional position errors of the corresponding debugging faces is determined as the preset constraint condition.

5. The method according to any one of claims 1 to 3, characterized in that: The target face is a face of a person wearing a mask, and the unobstructed portion is a portion of the target face that is not obstructed by the mask.

6. A face recognition device, characterized in that: The device comprises: A first acquisition module is configured to obtain face images acquired by multiple image acquisition devices in the same acquisition cycle, wherein the multiple image acquisition devices photograph the target face from different shooting angles, and part of the target face is blocked; A detection module is configured to detect facial feature points of unobstructed parts of a target face from the multiple face images; A first determination module is configured to determine the three-dimensional position information corresponding to the target facial feature points in the target face based on the facial feature points detected in each facial image and the device information of each image acquisition device; A second determination module is configured to determine a target three-dimensional face model corresponding to the target face based on an initial three-dimensional face model, preset constraints, and three-dimensional position information corresponding to the target face feature points, wherein the target three-dimensional face model includes three-dimensional position information corresponding to the face feature points of the occluded parts of the target face, the initial three-dimensional face model includes: a plurality of sample face models and their corresponding weight parameters, and the preset constraints are used to constrain the values ​​of the weight parameters corresponding to the plurality of sample face models; The second determining module comprises: A first determining unit is configured to determine first projection position information corresponding to each first reference facial feature point in each face image based on three-dimensional position information corresponding to the first reference facial feature point corresponding to each target facial feature point in each reference facial feature point in the initial three-dimensional face model and device information of each image acquisition device, wherein the reference facial feature points include: facial feature points of various parts of the face, and the same part of the face corresponding to the first reference facial feature point and the target facial feature point that have a corresponding relationship; A second determining unit is configured to determine second projection position information corresponding to each target facial feature point in each facial image based on the three-dimensional position information corresponding to the target facial feature point and device information of each image acquisition device; The third determination unit is configured to determine the target three-dimensional face model corresponding to the target face based on the second projection position information corresponding to each target face feature point, the first projection position information corresponding to each first reference face feature point and the preset constraint conditions.

7. The device according to claim 6, characterized in that The third determining unit is specifically configured to determine a first projection error corresponding to the target face based on the second projection position information corresponding to each target facial feature point, the first projection position information corresponding to each first reference facial feature point, and a preset constraint condition; Determine whether the first projection error does not exceed a preset error threshold; If it is determined that the first projection error does not exceed a preset error threshold, determining the current initial three-dimensional face model as a target three-dimensional face model corresponding to the target face; If it is determined that the first projection error exceeds a preset error threshold, adjusting current values ​​of weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain a current initial three-dimensional face model; Determine new first projection position information corresponding to each first reference facial feature point in each facial image based on new three-dimensional position information corresponding to the first reference facial feature point in the current initial three-dimensional facial model and device information of each image acquisition device; Based on the second projection position information corresponding to each target facial feature point, the new first projection position information corresponding to each first reference facial feature point and the preset constraints, determine the first projection error corresponding to the target face; return to determine whether the first projection error does not exceed the preset error threshold.

8. The device according to claim 6, characterized in that The second determination module is specifically configured to determine the three-dimensional position information corresponding to each second reference facial feature point corresponding to the target facial feature point based on the current values ​​of the multiple sample facial models and their corresponding weight parameters in the initial three-dimensional facial model, wherein the second reference facial feature points and the target facial feature points that have a corresponding relationship correspond to the same part of the face; Determine a first distance error corresponding to the target face based on the three-dimensional position information corresponding to each target facial feature point and the three-dimensional position information corresponding to each corresponding second reference facial feature point and preset constraints; Determine whether a first distance error corresponding to the target face does not exceed a first distance error threshold; If it is determined that the first distance error does not exceed a first distance error threshold, determining the current initial three-dimensional face model as a target three-dimensional face model corresponding to the target face; If it is determined that the first distance error exceeds a first distance error threshold, adjusting current values ​​of weight parameters corresponding to the multiple sample face models in the initial three-dimensional face model to obtain a current initial three-dimensional face model; Based on the adjusted values ​​of the multiple sample face models and their corresponding weight parameters in the current initial three-dimensional face model, determining new three-dimensional position information corresponding to each second reference face feature point; Based on the three-dimensional position information corresponding to each target facial feature point and the new three-dimensional position information corresponding to each corresponding second reference facial feature point and the preset constraints, determine the first distance error corresponding to the target face; return to determine whether the first distance error corresponding to the target face does not exceed the first distance error threshold.

Images