Face recognition method and device, storage medium and electronic equipment

Abstract

The specification discloses a face recognition method and device, a storage medium and an electronic device, wherein the method comprises: acquiring a face image containing a target face, the target face being a face selected for face recognition; performing risk assessment on the target face in the face image based on a pre-trained face recognition risk assessment model to obtain a risk assessment result corresponding to the target face; and if the risk assessment result indicates that the target face is safe for face recognition, performing face recognition on the target face.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a facial recognition method, device, storage medium, and electronic device. Background Technology

[0002] As facial recognition technology matures, its commercial applications are becoming increasingly widespread, such as in financial transactions, access control systems, and mobile terminals. When users use facial recognition devices in offline Internet of Things (IoT) locations, if the user is in an open public place, the device may collect facial images containing multiple faces and select one face from those images for recognition. However, this method of selecting a face for recognition may lead to the mistaken recognition of the wrong face. Summary of the Invention

[0003] This specification provides a facial recognition method, device, storage medium, and electronic device, which improves facial recognition security by assessing the risk of a selected target face in a facial image. The technical solution is as follows:

[0004] Firstly, embodiments of this specification provide a facial recognition method, the method comprising:

[0005] Acquire a facial image containing the target face, wherein the target face is the face selected for facial recognition;

[0006] The risk assessment of the target face in the face image is performed based on the pre-trained face recognition risk assessment model to obtain the risk assessment result corresponding to the target face.

[0007] If the risk assessment result indicates that the target face is safe for facial recognition, then facial recognition is performed on the target face.

[0008] Secondly, embodiments of this specification provide a method for training a facial recognition risk assessment model, the method comprising:

[0009] A training image dataset is established, which includes each sample face image, the corresponding depth face image, and the risk label corresponding to the sample face in the sample face image. The sample face is the face selected from the sample face images for face recognition.

[0010] The sample face image is input into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result output by the face recognition risk assessment model for the sample face image;

[0011] Based on the depth prediction image and the depth face image, the depth prediction loss value is calculated using a preset depth prediction loss function;

[0012] Based on the risk prediction results and the risk labels, the risk prediction loss value is calculated using a preset risk prediction loss function.

[0013] The model parameters of the facial recognition risk assessment model are updated based on the depth prediction loss value and the risk prediction loss value.

[0014] Determine whether the risk assessment model with updated parameters meets the preset convergence condition. If it does, stop training and obtain the trained risk assessment model. If it does not meet the condition, execute the step of inputting the sample face image into the risk assessment model.

[0015] Thirdly, embodiments of this specification provide a facial recognition device, the device comprising:

[0016] The image acquisition module is used to acquire a face image containing a target face, wherein the target face is the face selected for face recognition.

[0017] The risk assessment module is used to assess the risk of a target face in the face image based on a pre-trained face recognition risk assessment model, and obtain the risk assessment result corresponding to the target face.

[0018] The facial recognition module is used to perform facial recognition on the target face if the risk assessment result indicates that facial recognition of the target face is safe.

[0019] Fourthly, this specification provides a facial recognition risk assessment model training device, the device comprising:

[0020] The training set establishment module is used to establish a training image dataset, which includes each sample face image, the corresponding depth face image, and the risk label corresponding to the sample face in the sample face image. The sample face is the face selected from the sample face images for face recognition.

[0021] The model prediction module is used to input the sample face image into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result output by the face recognition risk assessment model for the sample face image;

[0022] The first loss calculation module is used to calculate the depth prediction loss value based on the depth prediction image and the depth face image using a preset depth prediction loss function.

[0023] The second loss calculation module is used to calculate the risk prediction loss value based on the risk prediction result and the risk label using a preset risk prediction loss function.

[0024] The model parameter update module is used to update the model parameters of the face recognition risk assessment model based on the depth prediction loss value and the risk prediction loss value.

[0025] The model convergence judgment module is used to determine whether the risk assessment model with updated parameters meets the preset convergence condition. If it does, training is stopped and the trained risk assessment model is obtained. If it does not meet the condition, the step of inputting the sample face image into the risk assessment model is executed.

[0026] Fifthly, embodiments of this specification provide a computer program product that stores at least one instruction adapted to be loaded by a processor and executed in accordance with the above-described method steps.

[0027] Sixthly, embodiments of this specification provide a storage medium storing a computer program adapted to be loaded by a processor and to execute the above-described method steps.

[0028] In a seventh aspect, embodiments of this specification provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the above-described method steps.

[0029] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0030] The facial recognition method provided in the embodiments of this specification first acquires a facial image containing the target face during the facial recognition process. The target face is the face selected for facial recognition. Then, a risk assessment is performed on the target face in the facial image based on a pre-trained facial recognition risk assessment model to obtain the risk assessment result corresponding to the target face. If the risk assessment result indicates that facial recognition of the target face is safe, facial recognition is performed on the target face. By performing facial recognition risk assessment on the selected target face in the facial image, facial recognition is only performed on the target face when the risk assessment result indicates that it is safe, thereby improving the security of facial recognition for users and enhancing the user's security experience. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 A flowchart illustrating a facial recognition method provided in an embodiment of this specification;

[0033] Figure 2 A flowchart illustrating a facial recognition method provided in an embodiment of this specification;

[0034] Figure 3 A flowchart illustrating a facial recognition method provided in an embodiment of this specification;

[0035] Figure 4 This is a model architecture diagram of a facial recognition risk assessment model provided in the embodiments of this specification;

[0036] Figure 5 A flowchart illustrating a facial recognition method provided in an embodiment of this specification;

[0037] Figure 6 This is a model architecture diagram of a facial recognition risk assessment model provided in the embodiments of this specification;

[0038] Figure 7 A flowchart illustrating a face recognition risk assessment model training method provided in the embodiments of this specification;

[0039] Figure 8 This is a schematic diagram of the structure of a facial recognition risk assessment model provided in the embodiments of this specification;

[0040] Figure 9 This is a schematic diagram of the structure of a facial recognition device provided in the embodiments of this specification;

[0041] Figure 10 This is a schematic diagram of the structure of a facial recognition device provided in the embodiments of this specification;

[0042] Figure 11 This is a schematic diagram of the structure of a face recognition risk assessment model training device provided in the embodiments of this specification;

[0043] Figure 12 This is a structural block diagram of an electronic device provided as an embodiment of this specification. Detailed Implementation

[0044] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0045] In the description of this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this specification based on the specific circumstances. Furthermore, in the description of this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0046] The following detailed description is provided in conjunction with embodiments of the examples in this specification. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims. The flowcharts shown in the accompanying drawings are merely illustrative and are not necessarily to be performed in accordance with the steps shown. For example, some steps are parallel and do not have a strict logical order; therefore, the actual execution order is variable.

[0047] Current IoT facial recognition devices generally integrate multiple sensor types, such as 2D vision and 3D vision, to support facial recognition systems. However, different hardware configurations are offered for different use cases. For example, in scenarios where the cost of the device is acceptable to the merchant, low-cost facial recognition devices often only include a primary 2D vision sensor and do not include other modal sensors. In this case, if multiple people are present in the captured image, the facial recognition device may select the wrong target face, resulting in the wrong user being recognized.

[0048] Based on this, the embodiments of this specification provide a face recognition method. In the face recognition process, a face image containing the target face is first acquired. The target face is the face selected for face recognition. Then, a risk assessment is performed on the target face in the face image based on a pre-trained face recognition risk assessment model to obtain the risk assessment result corresponding to the target face. If the risk assessment result indicates that the target face is safe for face recognition, then face recognition is performed on the target face. By performing face recognition risk assessment on the selected target face in the face image, face recognition is only performed on the target face when the risk assessment result indicates that it is safe. This avoids the problem of accidental face recognition by face recognition users, improves the security of face recognition for face recognition users, and enhances the user's security experience.

[0049] Please see Figure 1 This is a flowchart illustrating a facial recognition method provided in an embodiment of this specification. In the embodiments of this specification, the facial recognition method is applied to a facial recognition device or an electronic device equipped with a facial recognition device. The following will focus on... Figure 1 The process shown will be explained in detail. The facial recognition method may specifically include the following steps:

[0050] S102, Obtain a face image containing the target face, where the target face is the face selected for face recognition;

[0051] In the embodiments described in this specification, the facial recognition device acquires a facial image containing a target face captured by the facial recognition device. The target face is the face selected by the facial recognition device from the facial image for facial recognition. For example, when user A performs facial recognition, the facial recognition device acquires a facial image including user A's face. Due to the acquisition range, the facial image also includes user B's face. Therefore, the facial recognition device selects one target face from user A and user B for facial recognition.

[0052] It should be noted that in the embodiments of this specification, the facial recognition device can be a facial recognition equipment, including but not limited to mobile phones, computers, tablets, smart wearable devices, in-vehicle devices, IoT devices and other electronic devices, or it can be a module in an electronic device used to implement the facial recognition method.

[0053] S104, Based on the pre-trained face recognition risk assessment model, the risk assessment of the target face in the face image is performed to obtain the risk assessment result corresponding to the target face.

[0054] In the embodiments of this specification, the facial image of the selected target face is input into a trained facial recognition risk assessment model. The pre-trained facial recognition risk assessment model can perform facial recognition risk assessment on the target face in the facial image and generate a risk assessment result for the target face. The risk assessment result is used to indicate whether facial recognition of the target face in the facial image is safe.

[0055] Understandably, during facial recognition, the target face selected by the device may not be the same as the user's face that needs to be recognized. For example, in a supermarket, when many customers are queuing to pay with facial recognition, if the facial recognition device captures the face of user B when user A initiates payment, and user B's face is used as the target face for recognition, user B could suffer financial loss. Therefore, during facial recognition, a risk assessment model can be used to assess the risk of the target face selected by the device. When the risk assessment model deems the target face safe, it indicates that the target face is the correct user's face; when the risk assessment model deems the target face dangerous, it indicates that the target face is an incorrect user's face.

[0056] In the embodiments of this specification, the risk assessment of a target face in a face image based on a pre-trained face recognition risk assessment model to obtain the risk assessment result corresponding to the target face may include: generating a mask image corresponding to the target face based on the position of the target face in the face image, the mask image including a first region corresponding to the target face in the face image and other regions in the face image excluding the first region, and inputting the mask image and the face image into the face recognition risk assessment model for risk assessment to obtain the risk assessment result corresponding to the target face.

[0057] The mask image is used to distinguish the target face region from other regions in a face image.

[0058] Optionally, a face bounding box of the target face in the face image is first generated using a face recognition algorithm, and the mask image is generated based on the face bounding box and the outer border of the face image.

[0059] In the embodiments of this specification, the facial recognition risk assessment model includes a depth estimation coding network, a two-dimensional modality coding network, a feature fusion network, and a risk prediction network. The step of inputting a mask image and a face image into the facial recognition risk assessment model for risk assessment to obtain the risk assessment result corresponding to the target face may include: encoding the face image based on the depth estimation coding network in the facial recognition risk assessment model to obtain a depth estimation feature map corresponding to the face image; encoding the face image based on the two-dimensional modality coding network in the facial recognition risk assessment model to obtain a two-dimensional modality feature map corresponding to the face image; performing feature fusion processing on the depth estimation feature map, the two-dimensional modality feature map, and the mask image based on the feature fusion network in the facial recognition risk assessment model to obtain a fused feature map; and performing risk prediction on the fused feature map based on the risk prediction network in the facial recognition risk assessment model to obtain the risk assessment result corresponding to the target face.

[0060] In the embodiments of this specification, the facial recognition risk assessment model includes a depth estimation coding network, a two-dimensional modality coding network, a feature fusion network, and a risk prediction network. The feature fusion network includes a first feature fusion network and a second feature fusion network. The process of fusing the depth estimation feature map, the two-dimensional modality feature map, and the mask map using the feature fusion network in the facial recognition risk assessment model to obtain a fused feature map includes: fusing the depth estimation feature map and the two-dimensional modality feature map using the first feature fusion network to obtain a depth modality feature map; and fusing the depth modality feature map and the mask map using the second feature fusion network to obtain the fused feature map.

[0061] In the embodiments of this specification, after obtaining the fused feature map, feature enhancement processing is performed on the fused feature map based on the attention mechanism to obtain the enhanced fused features. Risk prediction is then performed on the enhanced fused feature map based on the risk prediction network in the face recognition risk assessment model to obtain the risk assessment result corresponding to the target face.

[0062] S106 If the risk assessment result indicates that the target face is safe for facial recognition, then facial recognition is performed on the target face.

[0063] In the embodiments of this specification, when the risk assessment result given by the face recognition risk assessment model indicates that the target face is safe for face recognition, it means that the target face is the face that needs to be recognized by face recognition, and then face recognition is performed on the target face.

[0064] Optionally, when the risk assessment result given by the facial recognition risk assessment model indicates that the target face is dangerous, it means that the target face is not the face that needs to be recognized by facial recognition, and continuing to perform facial recognition may cause property loss or credit security, and then outputs a warning message about the danger of facial recognition.

[0065] In the embodiments of this specification, during the facial recognition process, a facial image containing the target face is first acquired. The target face is the face selected for facial recognition. Then, a risk assessment is performed on the target face in the facial image based on a pre-trained facial recognition risk assessment model to obtain the risk assessment result corresponding to the target face. If the risk assessment result indicates that facial recognition of the target face is safe, then facial recognition is performed on the target face. By performing facial recognition risk assessment on the selected target face in the facial image, facial recognition is only performed on the target face when the risk assessment result indicates that it is safe. This avoids the problem of accidental facial recognition by users, improves the security of facial recognition for users, and enhances the user's security experience.

[0066] Please see Figure 2 This is a flowchart illustrating a face recognition method provided in an embodiment of this specification.

[0067] like Figure 2 As shown, the facial recognition method may include the following steps:

[0068] S202, Obtain a face image containing the target face, where the target face is the face selected for face recognition;

[0069] In the embodiments of this specification, step S202 is described in detail in another embodiment of this specification for step S102, and will not be repeated here.

[0070] S204, Generate a mask image corresponding to the target face based on the position of the target face in the face image. The mask image includes a first region corresponding to the target face in the face image and other regions in the face image other than the first region.

[0071] In the embodiments of this specification, a face bounding box of the target face in a face image is generated using a face recognition algorithm, and a mask image is generated based on the face bounding box and the outer border of the face image. The mask image is used to distinguish the target face region from other regions in the face image.

[0072] In one feasible implementation, a mask image is generated based on the position of the selected target face in the face image (face bounding box position). This can be achieved by first calculating the face bounding box width w = x2 - x1 and the face bounding box height h = y2 - y1 based on the target face's position (x1, y1, x2, y2) in the face image. Then, the radius R = max(w / 2, h / 2) of the circular attention region is determined. A circular region (the first filling region) with radius R centered at the face bounding box position ((x1+x2) / 2, (y1+y2) / 2) is filled with 1s, while other background regions (the second filling region) are filled with 0s.

[0073] Optionally, mask images of shapes such as ellipses and squares can be generated based on the position of the target face bounding box in the face image. The specific shape can be selected according to actual needs.

[0074] S206, Input the mask image and face image into the face recognition risk assessment model to conduct risk assessment and obtain the risk assessment result corresponding to the target face;

[0075] In the embodiments of this specification, a mask image and a face image used to distinguish the target face region and other regions in a face image are input into the face recognition risk assessment model for risk assessment, and the risk assessment result corresponding to the target face is obtained.

[0076] S208, If the risk assessment result indicates that the target face is safe for facial recognition, then facial recognition is performed on the target face.

[0077] In the embodiments of this specification, when the risk assessment result given by the face recognition risk assessment model indicates that the target face is safe for face recognition, it means that the target face is the face that needs to be recognized by face recognition, and then face recognition is performed on the target face.

[0078] S210, if the risk assessment result indicates that the target person's face is dangerous to use facial recognition, then output a warning message indicating that facial recognition is dangerous.

[0079] In the embodiments of this specification, when the risk assessment result given by the face recognition risk assessment model indicates that the target face is dangerous, it means that the target face is not the face that needs to be recognized by face recognition, and continuing to perform face recognition may cause property loss or credit security, and then outputs a face recognition danger warning message.

[0080] For example, in a supermarket, when a large number of customers are queuing to pay by facial recognition, if the facial recognition device captures the face of user B and selects user B's face as the target face, the facial recognition risk assessment model can assess the risk of the target face selected by the device. If the risk assessment result indicates that using user B's face as the target face for facial recognition payment is dangerous, then the system will output a message: "Facial recognition is dangerous. Please keep away from the facial recognition area."

[0081] In the embodiments of this specification, during the facial recognition process, a facial image containing the target face is first acquired. The target face is the face selected for facial recognition. Then, a mask image corresponding to the target face is generated based on the position of the target face in the facial image. The mask image and the facial image are input into the facial recognition risk assessment model for risk assessment, and a risk assessment result corresponding to the target face is obtained. If the risk assessment result indicates that facial recognition of the target face is safe, facial recognition is performed on the target face. If the risk assessment result indicates that facial recognition of the target face is dangerous, a warning message indicating dangerous facial recognition is output. By performing facial recognition risk assessment on the selected target face in the facial image, facial recognition is only performed on the target face when the risk assessment result indicates that it is safe, thus avoiding the problem of accidental facial recognition by the user. This improves the security of facial recognition for users and enhances the user's security experience.

[0082] Please see Figure 3 This is a flowchart illustrating a face recognition method provided in an embodiment of this specification.

[0083] like Figure 3 As shown, the facial recognition method may include the following steps:

[0084] S302, Obtain a face image containing the target face, where the target face is the face selected for face recognition;

[0085] In the embodiments of this specification, step S302 is described in detail in another embodiment of this specification for step S102, and will not be repeated here.

[0086] S304, Generate a mask image corresponding to the target face based on the position of the target face in the face image. The mask image includes a first region corresponding to the target face in the face image and other regions in the face image other than the first region.

[0087] In the embodiments of this specification, step S304 is described in detail in another embodiment of this specification for step S204, and will not be repeated here.

[0088] S306, based on the depth estimation coding network in the face recognition risk assessment model, the face image is encoded to obtain the depth estimation feature map corresponding to the face image;

[0089] It should be noted that, in the embodiments of this specification, the facial recognition risk assessment model includes a depth estimation coding network, a two-dimensional modality coding network, a feature fusion network, and a risk prediction network.

[0090] In the embodiments of this specification, the face image is first encoded based on the depth estimation coding network in the face recognition risk assessment model. The depth estimation coding network can extract depth features from the face image to obtain the depth estimation feature map corresponding to the face image.

[0091] The depth estimation encoding network can be a common convolutional neural network such as Resent, VGG, MobileNet, ShuffleNetV2, etc.

[0092] S308, based on the two-dimensional modal coding network in the face recognition risk assessment model, encodes the face image to obtain the two-dimensional modal feature map corresponding to the face image;

[0093] In the embodiments of this specification, the face image is encoded based on the two-dimensional modal coding network in the face recognition risk assessment model. The two-dimensional modal coding network can extract two-dimensional image features from the face image to obtain the two-dimensional modal feature map corresponding to the face image.

[0094] Among them, the two-dimensional modal coding network can be a common convolutional neural network such as Resent, VGG, MobileNet, ShuffleNetV2, etc.

[0095] S310, based on the feature fusion network in the face recognition risk assessment model, performs feature fusion processing on the depth estimation feature map, two-dimensional modal feature map and mask map to obtain the fused feature map;

[0096] In the embodiments of this specification, after obtaining the depth estimation feature map, the two-dimensional modal feature map, and the mask map, the feature fusion network in the face recognition risk assessment model can perform feature fusion on the depth estimation feature map, the two-dimensional modal feature map, and the mask map to obtain a fused feature map that fuses the three feature images.

[0097] It should be noted that when performing feature fusion on the depth estimation feature map, the two-dimensional modality feature map, and the mask map, the depth estimation feature map, the two-dimensional modality feature map, and the mask map are first connected based on the channel dimension. Then, the connected feature map is input into the feature fusion network for feature fusion to obtain the fused feature map.

[0098] Optionally, after obtaining the fused feature map, feature enhancement processing is performed on the fused feature map based on the region attention mechanism to obtain the feature-enhanced fused feature map.

[0099] Understandably, the region attention mechanism can enhance the features of different regions based on the regions divided in the mask image, and perform risk prediction based on the fused feature map after feature enhancement, which can yield more accurate risk assessment results.

[0100] S312, based on the risk prediction network in the face recognition risk assessment model, performs risk prediction on the fused feature map to obtain the risk assessment result corresponding to the target face;

[0101] In the embodiments of this specification, after generating the fused feature map, the risk prediction network performs risk prediction on the fused feature map to obtain the risk assessment result corresponding to the target face.

[0102] S314, If the risk assessment result indicates that the target face is safe for facial recognition, then facial recognition is performed on the target face.

[0103] In the embodiments of this specification, when the risk assessment result given by the face recognition risk assessment model indicates that the target face is safe for face recognition, it means that the target face is the face that needs to be recognized by face recognition, and then face recognition is performed on the target face.

[0104] S316 If the risk assessment result indicates that the target person's face is dangerous to use facial recognition, then output a warning message indicating that facial recognition is dangerous.

[0105] In the embodiments of this specification, when the risk assessment result given by the face recognition risk assessment model indicates that the target face is dangerous, it means that the target face is not the face that needs to be recognized by face recognition, and continuing to perform face recognition may cause property loss or credit security, and then outputs a face recognition danger warning message.

[0106] Please see Figure 4 This is a model architecture diagram of a facial recognition risk assessment model provided in the embodiments of this specification. Figure 4 As shown, the facial recognition risk assessment model includes a depth estimation coding network, a two-dimensional modality coding network, a feature fusion network, and a risk prediction network. When a face image is input into the model, the depth estimation coding network extracts coded features from the face image to obtain a depth estimation feature map. The two-dimensional modality coding network then extracts coded features from the face image to obtain a two-dimensional modality feature map. The feature fusion network then fuses the depth estimation feature map, the two-dimensional modality feature map, and a mask image generated from the face image to obtain a fused feature map. Finally, the risk prediction network performs risk prediction based on the fused feature map to obtain a safe or dangerous risk prediction result.

[0107] In the embodiments of this specification, during the face recognition process, a face image containing the target face is first acquired. The target face is the face selected for face recognition. Then, a mask image corresponding to the target face is generated based on the position of the target face in the face image. The mask image and the face image are input into the face recognition risk assessment model. First, the depth estimation coding network in the face recognition risk assessment model encodes the face image to obtain a depth estimation feature map corresponding to the face image. Then, the two-dimensional modal coding network in the face recognition risk assessment model encodes the face image to obtain a two-dimensional modal feature map corresponding to the face image. Finally, the feature fusion network in the face recognition risk assessment model processes the depth estimation feature map, The two-dimensional modal feature map and the mask map are fused to obtain a fused feature map. Finally, the risk prediction network performs risk prediction on the fused feature map to obtain the risk assessment result corresponding to the target face. If the risk assessment result indicates that the target face is safe for face recognition, then face recognition is performed on the target face. If the risk assessment result indicates that the target face is dangerous for face recognition, then a warning message about the danger of face recognition is output. By performing face recognition risk assessment on the selected target face in the face image, face recognition is only performed on the target face when the risk assessment result indicates that it is safe. This avoids the problem of accidental face recognition by face recognition users, improves the security of face recognition for face recognition users, and enhances the user's security experience.

[0108] Please see Figure 5 This is a flowchart illustrating a face recognition method provided in an embodiment of this specification.

[0109] like Figure 5 As shown, the facial recognition method may include the following steps:

[0110] S402, Obtain a face image containing the target face, where the target face is the face selected for face recognition;

[0111] In the embodiments of this specification, step S402 is described in detail in another embodiment of this specification for step S102, and will not be repeated here.

[0112] S404, Generate a mask image corresponding to the target face based on the position of the target face in the face image. The mask image includes a first region corresponding to the target face in the face image and other regions in the face image other than the first region.

[0113] In the embodiments of this specification, step S404 is described in detail in another embodiment of this specification for step S304, and will not be repeated here.

[0114] S406, based on the depth estimation coding network in the face recognition risk assessment model, the face image is encoded to obtain the depth estimation feature map corresponding to the face image;

[0115] It should be noted that, in the embodiments of this specification, the face recognition risk assessment model includes a depth estimation coding network, a two-dimensional modality coding network, a first feature fusion network, a second feature fusion network, and a risk prediction network.

[0116] In the embodiments of this specification, the face image is first encoded based on the depth estimation coding network in the face recognition risk assessment model. The depth estimation coding network can extract depth features from the face image to obtain the depth estimation feature map corresponding to the face image.

[0117] The depth estimation encoding network can be a common convolutional neural network such as Resent, VGG, MobileNet, ShuffleNetV2, etc.

[0118] S408, based on the two-dimensional modal coding network in the face recognition risk assessment model, encodes the face image to obtain the two-dimensional modal feature map corresponding to the face image;

[0119] In the embodiments of this specification, the face image is encoded based on the two-dimensional modal coding network in the face recognition risk assessment model. The two-dimensional modal coding network can extract two-dimensional image features from the face image to obtain the two-dimensional modal feature map corresponding to the face image.

[0120] Among them, the two-dimensional modal coding network can be a common convolutional neural network such as Resent, VGG, MobileNet, ShuffleNetV2, etc.

[0121] S410, Based on the first feature fusion network, feature fusion processing is performed on the depth estimation feature map and the two-dimensional modal feature map to obtain the depth modal feature map;

[0122] In the embodiments of this specification, after obtaining the depth estimation feature map and the two-dimensional modal feature map, the depth estimation feature map and the two-dimensional modal feature map are first connected together according to the channel dimension. Then, the connected feature map is input into the first feature fusion network for feature fusion to obtain the depth modal feature map after fusing the depth estimation feature map and the two-dimensional modal feature map.

[0123] S412, Based on the second feature fusion network, feature fusion processing is performed on the deep modality feature map and the mask map to obtain the fused feature map;

[0124] In the embodiments of this specification, the deep modality feature map and the mask map are fused based on the second feature fusion network in the face recognition risk assessment model to obtain a fused feature map.

[0125] It should be noted that the mask image and the depth modality feature image have the same resolution. If the mask image and the depth modality feature image generated in step S404 have different resolutions, before performing feature fusion processing on the depth modality feature image and the mask image based on the second feature fusion network to obtain the fused feature image, a mask image with the same resolution as the depth modality feature image is generated by nearest neighbor sampling based on the mask image generated in step S404. Then, the step of performing feature fusion processing on the depth modality feature image and the mask image based on the second feature fusion network to obtain the fused feature image is executed.

[0126] Optionally, after obtaining the fused feature map, feature enhancement processing is performed on the fused feature map based on the region attention mechanism to obtain the feature-enhanced fused feature map.

[0127] Understandably, the region attention mechanism can enhance the features of different regions based on the regions divided in the mask image, and perform risk prediction based on the fused feature map after feature enhancement, which can yield more accurate risk assessment results.

[0128] S414, based on the risk prediction network in the face recognition risk assessment model, risk prediction is performed on the fused feature map to obtain the risk assessment result corresponding to the target face;

[0129] S416 If the risk assessment result indicates that the target face is safe for facial recognition, then facial recognition is performed on the target face.

[0130] S418, if the risk assessment result indicates that the target person's face is dangerous to use facial recognition, then output a warning message indicating that facial recognition is dangerous.

[0131] Please see Figure 6 This is a model architecture diagram of a facial recognition risk assessment model provided in the embodiments of this specification. Figure 6 As shown, the facial recognition risk assessment model includes a depth estimation coding network, a two-dimensional modality coding network, a first feature fusion network, a second feature fusion network, and a risk prediction network. When a face image is input into the model, the depth estimation coding network extracts coded features from the face image to obtain a depth estimation feature map. The two-dimensional modality coding network then extracts coded features from the face image to obtain a two-dimensional modality feature map. The first feature fusion network then fuses the depth estimation feature map and the two-dimensional modality feature map to obtain a depth modality feature map. The second feature fusion network then fuses the depth modality feature map with a mask image generated from the face image to obtain a fused feature map. Finally, the risk prediction network performs risk prediction based on the fused feature map to obtain a safe or dangerous risk prediction result.

[0132] In the embodiments of this specification, during the face recognition process, firstly, a face image containing the target face is acquired. The target face is the face selected for face recognition. Then, a mask image corresponding to the target face is generated based on the position of the target face in the face image. The face image is then encoded by the depth estimation coding network in the face recognition risk assessment model to obtain a depth estimation feature map corresponding to the face image. The face image is then encoded by the two-dimensional modality coding network in the face recognition risk assessment model to obtain a two-dimensional modality feature map corresponding to the face image. Then, the depth estimation feature map and the two-dimensional modality feature map are fused based on the first feature fusion network to obtain a depth modality feature map. Finally, the two-dimensional modality feature map is fused based on the second feature fusion network. The network performs feature fusion processing on the deep modal feature map and the mask image to obtain a fused feature map. Finally, the risk prediction network performs risk prediction on the fused feature map to obtain the risk assessment result corresponding to the target face. If the risk assessment result indicates that the target face is safe for face recognition, then face recognition is performed on the target face. If the risk assessment result indicates that the target face is dangerous for face recognition, then a warning message about the danger of face recognition is output. By performing face recognition risk assessment on the selected target face in the face image, face recognition is only performed on the target face when the risk assessment result indicates that it is safe. This avoids the problem of accidental face recognition by face recognition users, improves the security of face recognition for face recognition users, and enhances the user's security experience.

[0133] Please see Figure 7 This is a flowchart illustrating a face recognition risk assessment model training method provided in an embodiment of this specification. Figure 7 As shown, the training method for the facial recognition risk assessment model may include the following steps:

[0134] S502, Establish a training image dataset. The training image dataset includes each sample face image, the corresponding depth face image, and the risk label corresponding to the sample face in the sample face image. The sample face is the face selected from the sample face images for face recognition.

[0135] In the embodiments of this specification, a training image dataset is first created, and sample face images and corresponding depth face images are collected when each sample face is used for face recognition. Risk labels are then added to selected sample faces in each sample face image.

[0136] The risk label can be either a safety label or a danger label. Based on the different risk labels, the sample face images can be divided into positive sample images and negative sample images.

[0137] S504, Input the sample face image into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result output by the face recognition risk assessment model for the sample face image;

[0138] In the embodiments of this specification, the sample face image is input into the completed face recognition risk assessment model to obtain the depth prediction image and risk prediction result output by the sample face image of the face recognition risk assessment model.

[0139] In the embodiments of this specification, a sample mask map corresponding to the sample face is generated based on the position of the sample face in the sample face image. The sample mask map includes a first region corresponding to the sample face in the sample face image and other regions in the sample face image other than the first region. The sample mask map and the sample face image are input into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result corresponding to the sample face image.

[0140] Optionally, the sample mask image and sample face image are input into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result corresponding to the sample face image. This includes: encoding the sample face image based on the depth estimation encoding network in the face recognition risk assessment model to obtain the sample depth estimation feature map corresponding to the sample face image; encoding the sample face image based on the two-dimensional modality encoding network in the face recognition risk assessment model to obtain the sample two-dimensional modality feature map corresponding to the sample face image; performing depth prediction based on the sample depth estimation feature map based on the depth prediction network in the face recognition risk assessment model to obtain the depth prediction image; performing feature fusion processing on the sample depth estimation feature map, sample two-dimensional modality feature map and sample mask image based on the feature fusion network in the face recognition risk assessment model to obtain the sample fused feature map; and performing risk prediction based on the sample fused feature map based on the risk prediction network in the face recognition risk assessment model to obtain the risk prediction result corresponding to the sample face image.

[0141] Optionally, feature fusion processing is performed on the sample depth estimation feature map, sample two-dimensional modal feature map, and sample mask map based on the feature fusion network in the face recognition risk assessment model to obtain a sample fusion feature map, including: performing feature fusion processing on the sample depth estimation feature map and sample two-dimensional modal feature map based on the first feature fusion network to obtain a sample depth modal feature map, and performing feature fusion processing on the sample depth modal feature map and sample mask map based on the second feature fusion network to obtain a sample fusion feature map.

[0142] Please see Figure 8 This is a structural diagram of a facial recognition risk assessment model provided in an embodiment of this specification. Figure 8As shown, the facial recognition risk assessment model includes a depth estimation coding network, a two-dimensional modality coding network, a first feature fusion network, a second feature fusion network, a depth prediction network, and a risk prediction network. After obtaining the sample face image and sample mask image, the depth estimation coding network encodes the sample face image to obtain a sample depth estimation feature map. The depth prediction network then predicts the depth of the sample depth estimation feature map to obtain a depth prediction image. The two-dimensional modality coding network encodes the sample face image to obtain a sample two-dimensional modality feature map. The first feature fusion network fuses the sample two-dimensional modality feature map and the sample depth estimation coding feature map to obtain a sample depth modality feature map. Then, the second feature fusion network fuses the sample depth modality feature map and the sample mask image to obtain a sample fused feature map. Finally, the risk prediction network performs risk prediction on the sample fused feature map to obtain the risk prediction result corresponding to the sample face image.

[0143] S506, based on the depth prediction image and the depth face image, calculates the depth prediction loss value through a preset depth prediction loss function;

[0144] S508, based on the risk prediction results and risk labels, calculates the risk prediction loss value through a preset risk prediction loss function;

[0145] S510 updates the model parameters of the face recognition risk assessment model based on the depth prediction loss value and the risk prediction loss value;

[0146] In the embodiments of this specification, the network parameters of the depth estimation coding network are updated based on the depth prediction loss value, and the model parameters of the face recognition risk assessment model are updated based on the risk prediction loss value.

[0147] It is understandable that updating the parameters of the depth estimation coding network by using the depth prediction loss value calculated from the depth prediction image and the depth face image can update the model parameters of the depth estimation coding network, enabling the depth estimation coding network to extract the depth features of the face image more accurately, thereby improving the accuracy of the model prediction.

[0148] S512, determine whether the risk assessment model with updated parameters meets the preset convergence condition. If it does, stop training and obtain the trained risk assessment model. If it does not meet the condition, proceed to step S504.

[0149] In the embodiments of this specification, a training image dataset is established, which includes sample face images, corresponding depth face images, and risk labels for the sample faces in the sample face images. The sample faces are the faces selected for face recognition in the sample face images. The sample face images are input into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result output by the face recognition risk assessment model for the sample face images. Based on the depth prediction image and the depth face image, a depth prediction loss value is calculated using a preset depth prediction loss function. Based on the risk prediction result and the risk label, a risk prediction loss value is calculated using a preset risk prediction loss function. The model parameters of the face recognition risk assessment model are updated based on the depth prediction loss value and the risk prediction loss value. By introducing the depth estimation task into the end-to-end learning network, the face recognition risk assessment model can achieve accurate risk prediction by performing depth prediction on face images and based on the depth estimation feature map.

[0150] Please see Figure 9 This is a structural schematic diagram of a facial recognition device provided in an embodiment of this specification. Figure 9 As shown, the facial recognition device 1 can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the facial recognition device 1 includes an image acquisition module 11, a risk assessment module 12, and a facial recognition module 13, specifically including:

[0151] Image acquisition module 11 is used to acquire a face image containing a target face, wherein the target face is the face selected for face recognition.

[0152] Risk assessment module 12 is used to assess the risk of a target face in the face image based on a pre-trained face recognition risk assessment model, and obtain the risk assessment result corresponding to the target face.

[0153] The face recognition module 13 is used to perform face recognition on the target face if the risk assessment result indicates that the face recognition of the target face is safe.

[0154] Optional, please see Figure 10 This is a structural schematic diagram of a facial recognition device provided in an embodiment of this specification. Figure 10 As shown, the facial recognition device also includes a risk warning module 14, used for:

[0155] If the risk assessment result indicates that the target face is at risk during facial recognition, a warning message indicating the risk of facial recognition will be output.

[0156] Optionally, the risk assessment module 12 is specifically used for:

[0157] A mask image corresponding to the target face is generated based on the position of the target face in the face image. The mask image includes a first region corresponding to the target face in the face image and other regions in the face image other than the first region.

[0158] The mask image and the face image are input into the face recognition risk assessment model for risk assessment to obtain the risk assessment result corresponding to the target face.

[0159] Optionally, when the risk assessment module performs the step of inputting the mask image and the face image into the face recognition risk assessment model to perform risk assessment and obtain the risk assessment result corresponding to the target face, it is specifically used for:

[0160] The face image is encoded using the depth estimation coding network in the face recognition risk assessment model to obtain the depth estimation feature map corresponding to the face image.

[0161] The face image is encoded using the two-dimensional modal coding network in the face recognition risk assessment model to obtain the two-dimensional modal feature map corresponding to the face image.

[0162] Based on the feature fusion network in the facial recognition risk assessment model, feature fusion processing is performed on the depth estimation feature map, the two-dimensional modality feature map, and the mask map to obtain a fused feature map;

[0163] The risk prediction network in the face recognition risk assessment model is used to predict the risk of the fused feature map, thereby obtaining the risk assessment result corresponding to the target face.

[0164] Optionally, when the risk assessment module performs feature fusion processing on the depth estimation feature map, the two-dimensional modality feature map, and the mask map based on the feature fusion network in the facial recognition risk assessment model to obtain a fused feature map, it is specifically used for:

[0165] Based on the first feature fusion network, feature fusion processing is performed on the depth estimation feature map and the two-dimensional modal feature map to obtain a depth modal feature map;

[0166] The deep modality feature map and the mask map are fused based on the second feature fusion network to obtain the fused feature map.

[0167] Optionally, the risk assessment module is also used for:

[0168] The fused feature map is enhanced by an attention mechanism to obtain the enhanced fused feature map.

[0169] In the embodiments of this specification, during the facial recognition process, a facial image containing the target face is first acquired. The target face is the face selected for facial recognition. Then, a risk assessment is performed on the target face in the facial image based on a pre-trained facial recognition risk assessment model to obtain the risk assessment result corresponding to the target face. If the risk assessment result indicates that facial recognition of the target face is safe, then facial recognition is performed on the target face. By performing facial recognition risk assessment on the selected target face in the facial image, facial recognition is only performed on the target face when the risk assessment result indicates that it is safe. This avoids the problem of accidental facial recognition by users, improves the security of facial recognition for users, and enhances the user's security experience.

[0170] It should be noted that the facial recognition device provided in the above embodiments is only illustrated by the division of the above functional modules when performing the facial recognition method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the facial recognition device and the facial recognition method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0171] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0172] Please see Figure 11 This is a schematic diagram of the structure of a facial recognition risk assessment model training device provided in an embodiment of this specification. Figure 11 As shown, the facial recognition risk assessment model training device 2 can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the facial recognition risk assessment model training device 2 includes a training set establishment module 21, a model prediction module 22, a first loss calculation module 23, a second loss calculation module 24, a model parameter update module 25, and a model convergence judgment module 26, specifically including:

[0173] The training set establishment module 21 is used to establish a training image dataset, which includes each sample face image, the corresponding depth face image, and the risk label corresponding to the sample face in the sample face image. The sample face is the face selected from the sample face images for face recognition.

[0174] The model prediction module 22 is used to input the sample face image into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result output by the face recognition risk assessment model for the sample face image;

[0175] The first loss calculation module 23 is used to calculate the depth prediction loss value based on the depth prediction image and the depth face image by using a preset depth prediction loss function.

[0176] The second loss calculation module 24 is used to calculate the risk prediction loss value based on the risk prediction result and the risk label using a preset risk prediction loss function.

[0177] The model parameter update module 25 is used to update the model parameters of the face recognition risk assessment model based on the depth prediction loss value and the risk prediction loss value.

[0178] The model convergence judgment module 26 is used to determine whether the risk assessment model with updated parameters meets the preset convergence condition. If it does, training is stopped and the trained risk assessment model is obtained. If it does not meet the condition, the step of inputting the sample face image into the risk assessment model is executed.

[0179] Optionally, the model prediction module 22 is specifically used for:

[0180] A sample mask map corresponding to the sample face is generated based on the position of the sample face in the sample face image. The sample mask map includes a first region corresponding to the sample face in the sample face image and other regions in the sample face image other than the first region.

[0181] The sample mask image and the sample face image are input into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result corresponding to the sample face image.

[0182] Optionally, when the model prediction module 22 performs the step of inputting the sample mask image and the sample face image into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result corresponding to the sample face image, it is specifically used for:

[0183] The sample face image is encoded based on the depth estimation coding network in the face recognition risk assessment model to obtain the sample depth estimation feature map corresponding to the sample face image;

[0184] The sample face image is encoded using the two-dimensional modal coding network in the face recognition risk assessment model to obtain the sample two-dimensional modal feature map corresponding to the sample face image.

[0185] Based on the depth prediction network in the facial recognition risk assessment model, the depth estimation feature map of the sample is used to predict the depth, and the depth prediction image is obtained.

[0186] Based on the feature fusion network in the facial recognition risk assessment model, the sample depth estimation feature map, the sample two-dimensional modal feature map, and the sample mask map are subjected to feature fusion processing to obtain the sample fusion feature map;

[0187] The risk prediction network in the face recognition risk assessment model is used to predict the risk of the sample fusion feature map, and the risk prediction result corresponding to the sample face image is obtained.

[0188] Optionally, when the model prediction module 22 performs feature fusion processing on the sample depth estimation feature map, the sample two-dimensional modality feature map, and the sample mask map based on the feature fusion network in the face recognition risk assessment model to obtain the sample fused feature map, it is specifically used for:

[0189] Based on the first feature fusion network, feature fusion processing is performed on the sample depth estimation feature map and the sample two-dimensional modal feature map to obtain the sample depth modal feature map;

[0190] The sample deep modal feature map and the sample mask map are fused based on the second feature fusion network to obtain the sample fused feature map.

[0191] In the embodiments of this specification, a training image dataset is established, which includes sample face images, corresponding depth face images, and risk labels for the sample faces in the sample face images. The sample faces are the faces selected for face recognition in the sample face images. The sample face images are input into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result output by the face recognition risk assessment model for the sample face images. Based on the depth prediction image and the depth face image, a depth prediction loss value is calculated using a preset depth prediction loss function. Based on the risk prediction result and the risk label, a risk prediction loss value is calculated using a preset risk prediction loss function. The model parameters of the face recognition risk assessment model are updated based on the depth prediction loss value and the risk prediction loss value. By introducing the depth estimation task into the end-to-end learning network, the face recognition risk assessment model can achieve accurate risk prediction by performing depth prediction on face images and based on the depth estimation feature map.

[0192] It should be noted that the facial recognition risk assessment model training device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the facial recognition risk assessment model training method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the facial recognition risk assessment model training device and the facial recognition risk assessment model training method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0193] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0194] This specification also provides an embodiment of a computer storage medium that can store multiple instructions adapted to be loaded and executed by a processor as described above. Figures 1 to 8 The face recognition method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 1 to 8 The specific details of the illustrated embodiments will not be elaborated here.

[0195] This specification also provides a computer program product that stores at least one instruction, said at least one instruction being loaded and executed by the processor as described above. Figures 1 to 8 The face recognition method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 1 to 8 The specific details of the illustrated embodiments will not be elaborated here.

[0196] Please refer to Figure 12 This is a structural block diagram of an electronic device provided in an embodiment of this specification. The electronic device in this specification may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 can be connected via the bus 150.

[0197] Processor 110 may include one or more processing cores. Processor 110 connects to various parts of the terminal using various interfaces and lines, and performs various functions and processes data of terminal 100 by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 110 may integrate one or more of the following: central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 110 and may be implemented separately using a communication chip.

[0198] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 may include non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets.

[0199] The input device 130 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 140 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In this embodiment, the input device 130 can be a temperature sensor to obtain the operating temperature of the terminal. The output device 140 can be a speaker to output audio signals.

[0200] In addition, those skilled in the art will understand that the structure of the terminal shown in the above figures does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the terminal may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WIFI) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0201] In the embodiments of this specification, the executing entity for each step can be the terminal described above. Optionally, the executing entity for each step is the terminal's operating system. The operating system can be Android, iOS, or other operating systems; this specification does not limit this.

[0202] exist Figure 12 In the electronic device, the processor 110 can be used to call the face recognition interaction program stored in the memory 120 and execute it to implement the face recognition method as described in the various method embodiments of this specification.

[0203] In the embodiments of this specification, during the facial recognition process, a facial image containing the target face is first acquired. The target face is the face selected for facial recognition. Then, a risk assessment is performed on the target face in the facial image based on a pre-trained facial recognition risk assessment model to obtain the risk assessment result corresponding to the target face. If the risk assessment result indicates that facial recognition of the target face is safe, then facial recognition is performed on the target face. By performing facial recognition risk assessment on the selected target face in the facial image, facial recognition is only performed on the target face when the risk assessment result indicates that it is safe. This avoids the problem of accidental facial recognition by users, improves the security of facial recognition for users, and enhances the user's security experience.

[0204] Those skilled in the art will clearly understand that the technical solutions in this specification can be implemented using software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware capable of independently performing or cooperating with other components to perform a specific function. Hardware may include, for example, a Field-Programmable Gate Array (FPGA), an Integrated Circuit (IC), etc.

[0205] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this specification is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this specification. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this specification.

[0206] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0207] In the several embodiments provided in this specification, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0208] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0209] Furthermore, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0210] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0211] The foregoing descriptions are merely exemplary embodiments of this specification and should not be construed as limiting the scope of this specification. Any equivalent changes and modifications made in accordance with the teachings of this specification shall still fall within the scope of this specification. Other embodiments of this specification will be readily apparent to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this specification are defined by the claims.

Claims

1. A facial recognition method, the method comprising: Acquire a facial image containing the target face, wherein the target face is the face selected for facial recognition; The risk assessment of the target face in the face image is performed based on the pre-trained face recognition risk assessment model to obtain the risk assessment result corresponding to the target face. If the risk assessment result indicates that the target face is safe for facial recognition, then facial recognition is performed on the target face. The pre-trained facial recognition risk assessment model performs risk assessment on the target face in the facial image, and obtains the risk assessment result corresponding to the target face, including: A mask image corresponding to the target face is generated based on the position of the target face in the face image. The mask image includes a first region corresponding to the target face in the face image and other regions in the face image other than the first region. The mask image and the face image are input into the face recognition risk assessment model for risk assessment to obtain the risk assessment result corresponding to the target face.

2. The method according to claim 1, further comprising: If the risk assessment result indicates that the target face is at risk during facial recognition, a warning message indicating the risk of facial recognition will be output.

3. The method according to claim 1, wherein inputting the mask image and the face image into the face recognition risk assessment model for risk assessment to obtain the risk assessment result corresponding to the target face includes: The face image is encoded using the depth estimation coding network in the face recognition risk assessment model to obtain the depth estimation feature map corresponding to the face image. The face image is encoded using the two-dimensional modal coding network in the face recognition risk assessment model to obtain the two-dimensional modal feature map corresponding to the face image. Based on the feature fusion network in the facial recognition risk assessment model, feature fusion processing is performed on the depth estimation feature map, the two-dimensional modality feature map, and the mask map to obtain a fused feature map; The risk prediction network in the face recognition risk assessment model is used to predict the risk of the fused feature map, thereby obtaining the risk assessment result corresponding to the target face.

4. The method according to claim 3, wherein the feature fusion network comprises a first feature fusion network and a second feature fusion network; The feature fusion network based on the facial recognition risk assessment model performs feature fusion processing on the depth estimation feature map, the two-dimensional modal feature map, and the mask map to obtain a fused feature map, including: Based on the first feature fusion network, feature fusion processing is performed on the depth estimation feature map and the two-dimensional modal feature map to obtain a depth modal feature map; The deep modality feature map and the mask map are fused based on the second feature fusion network to obtain the fused feature map.

5. The method according to claim 3, further comprising, after obtaining the fused feature map: The fused feature map is enhanced by an attention mechanism to obtain the enhanced fused feature map.

6. A method for training a facial recognition risk assessment model, comprising: A training image dataset is established, which includes each sample face image, the corresponding depth face image, and the risk label corresponding to the sample face in the sample face image. The sample face is the face selected from the sample face images for face recognition. The sample face image is input into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result output by the face recognition risk assessment model for the sample face image; Based on the depth prediction image and the depth face image, the depth prediction loss value is calculated using a preset depth prediction loss function; Based on the risk prediction results and the risk labels, the risk prediction loss value is calculated using a preset risk prediction loss function. The model parameters of the facial recognition risk assessment model are updated based on the depth prediction loss value and the risk prediction loss value. Determine whether the risk assessment model with updated parameters meets the preset convergence condition. If it does, stop training and obtain the trained risk assessment model. If it does not meet the condition, execute the step of inputting the sample face image into the risk assessment model.

7. The method according to claim 6, wherein inputting the sample face image into the risk assessment model to obtain the depth prediction image and risk prediction result output by the face recognition risk assessment model for the sample face image includes: A sample mask map corresponding to the sample face is generated based on the position of the sample face in the sample face image. The sample mask map includes a first region corresponding to the sample face in the sample face image and other regions in the sample face image other than the first region. The sample mask image and the sample face image are input into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result corresponding to the sample face image.

8. The method according to claim 7, wherein inputting the sample mask image and the sample face image into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result corresponding to the sample face image includes: The sample face image is encoded based on the depth estimation coding network in the face recognition risk assessment model to obtain the sample depth estimation feature map corresponding to the sample face image; The sample face image is encoded using the two-dimensional modal coding network in the face recognition risk assessment model to obtain the sample two-dimensional modal feature map corresponding to the sample face image. Based on the depth prediction network in the facial recognition risk assessment model, the depth estimation feature map of the sample is used to predict the depth, and the depth prediction image is obtained. Based on the feature fusion network in the facial recognition risk assessment model, the sample depth estimation feature map, the sample two-dimensional modal feature map, and the sample mask map are subjected to feature fusion processing to obtain the sample fusion feature map; The risk prediction network in the face recognition risk assessment model is used to predict the risk of the sample fusion feature map, and the risk prediction result corresponding to the sample face image is obtained.

9. The method according to claim 8, wherein the feature fusion network comprises a first feature fusion network and a second feature fusion network; The feature fusion network based on the facial recognition risk assessment model performs feature fusion processing on the sample depth estimation feature map, the sample two-dimensional modal feature map, and the sample mask map to obtain a sample fused feature map, including: Based on the first feature fusion network, feature fusion processing is performed on the sample depth estimation feature map and the sample two-dimensional modal feature map to obtain the sample depth modal feature map; The sample deep modal feature map and the sample mask map are fused based on the second feature fusion network to obtain the sample fused feature map.

10. A facial recognition device, comprising: The image acquisition module is used to acquire a face image containing a target face, wherein the target face is the face selected for face recognition. The risk assessment module is used to assess the risk of a target face in the face image based on a pre-trained face recognition risk assessment model, and obtain the risk assessment result corresponding to the target face. A face recognition module is used to perform face recognition on the target face if the risk assessment result indicates that face recognition of the target face is safe. The risk assessment module is used for: A mask image corresponding to the target face is generated based on the position of the target face in the face image. The mask image includes a first region corresponding to the target face in the face image and other regions in the face image other than the first region. The mask image and the face image are input into the face recognition risk assessment model for risk assessment to obtain the risk assessment result corresponding to the target face.

11. A training device for a facial recognition risk assessment model, comprising: The training set establishment module is used to establish a training image dataset, which includes each sample face image, the corresponding depth face image, and the risk label corresponding to the sample face in the sample face image. The sample face is the face selected from the sample face images for face recognition. The model prediction module is used to input the sample face image into the face recognition risk assessment model to obtain the depth prediction image and risk prediction result output by the face recognition risk assessment model for the sample face image; The first loss calculation module is used to calculate the depth prediction loss value based on the depth prediction image and the depth face image using a preset depth prediction loss function. The second loss calculation module is used to calculate the risk prediction loss value based on the risk prediction result and the risk label using a preset risk prediction loss function. The model parameter update module is used to update the model parameters of the face recognition risk assessment model based on the depth prediction loss value and the risk prediction loss value. The model convergence judgment module is used to determine whether the risk assessment model with updated parameters meets the preset convergence condition. If it does, training is stopped and the trained risk assessment model is obtained. If it does not meet the condition, the step of inputting the sample face image into the risk assessment model is executed.

12. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5 or 6 to 9.

13. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as claimed in any one of claims 1 to 5 or 6 to 9.

14. A computer program product having at least one instruction stored thereon, characterized in that, When the at least one instruction is executed by the processor, it implements the steps of the method according to any one of claims 1 to 5 or 6 to 9.

Images