Living body detection model training method and device, storage medium and equipment

Abstract

The specification discloses a live body detection model training method and device, a storage medium and equipment, wherein the method comprises the following steps: constructing a training image dataset, each training image data in the training image dataset comprises at least two modal images and standard results of whether the modal images are live bodies, inputting each modal image in the training image data into a live body detection model, performing feature extraction processing on each modal image by using a feature extraction network to obtain image features corresponding to each modal image respectively, performing feature fusion processing on each image feature by using a feature fusion network to obtain fused image features, performing live body prediction on the fused image features by using a feature prediction network to obtain live body prediction results corresponding to the training image data, and performing supervised training on the live body detection model based on a preset loss function, the standard results corresponding to the training image data and the live body prediction results to obtain a trained live body detection model.

Description

Technical Field

[0001] This specification relates to the field of biometric technology, and in particular to a method, apparatus, storage medium and device for training a liveness detection model. Background Technology

[0002] As facial recognition technology matures, its commercial applications are becoming increasingly widespread, such as in financial transactions, access control systems, and mobile terminals. However, faces are easily replicated using photographs, videos, models, or masks, making the impersonation of legitimate users a significant threat to the security of facial recognition and authentication systems. To prevent malicious actors from forging and stealing others' biometric features for identity authentication, biometric systems must have liveness detection capabilities. Summary of the Invention

[0003] This specification provides a method, apparatus, storage medium, and device for training a liveness detection model, which can obtain a liveness detection model with high accuracy. The technical solution is as follows:

[0004] Firstly, embodiments of this specification provide a method for training a liveness detection model, the method comprising:

[0005] Construct a training image dataset, wherein each training image in the training image dataset includes at least two modalities of images and a standard result of whether the modality of the image is a live organism;

[0006] Each modal image in the training image data is input into the liveness detection model, and the feature extraction network is used to perform feature extraction processing on each modal image to obtain the image features corresponding to each modal image.

[0007] The feature fusion network is used to perform feature fusion processing on each of the image features to obtain fused image features;

[0008] The feature prediction network is used to perform liveness prediction on the fused image features to obtain the liveness prediction result corresponding to the training image data;

[0009] The liveness detection model is trained under supervision based on a preset loss function, the standard results corresponding to the training image data, and the liveness prediction results. The model parameters of the liveness detection model are iteratively updated until the liveness detection model converges, resulting in a trained liveness detection model.

[0010] Secondly, embodiments of this specification provide a method for detecting live organisms, the method comprising:

[0011] Acquire at least two modal images of the face to be detected;

[0012] The at least two modal images are input into the trained liveness detection model obtained by the above-described liveness detection model training method, and the liveness detection result is output.

[0013] Thirdly, embodiments of this specification provide a liveness detection model training device, the device comprising:

[0014] The dataset construction module is used to construct a training image dataset, wherein each training image in the training image dataset includes at least two modal images and a standard result of whether the modal image is a live organism;

[0015] The feature extraction module is used to input each modal image in the training image data into the liveness detection model, and use the feature extraction network to perform feature extraction processing on each modal image to obtain the image features corresponding to each modal image.

[0016] The feature fusion module is used to perform feature fusion processing on each of the image features using the feature fusion network to obtain fused image features;

[0017] The liveness prediction module is used to perform liveness prediction on the fused image features using the feature prediction network to obtain the liveness prediction result corresponding to the training image data.

[0018] The parameter update module is used to supervise the training of the liveness detection model based on a preset loss function, the standard results corresponding to the training image data, and the liveness prediction results, and to iteratively update the model parameters of the liveness detection model until the liveness detection model converges, thus obtaining the trained liveness detection model.

[0019] Fourthly, embodiments of this specification provide a liveness detection device, the device comprising:

[0020] The image acquisition module is used to acquire at least two modal images of the face to be detected;

[0021] The liveness detection module is used to input the at least two modal images into the trained liveness detection model obtained by the above-described liveness detection model training method, and output the liveness detection result.

[0022] Fifthly, embodiments of this specification provide a computer program product that stores at least one instruction adapted to be loaded by a processor and executed in accordance with the above-described method steps.

[0023] Sixthly, embodiments of this specification provide a storage medium storing a computer program adapted to be loaded by a processor and to execute the above-described method steps.

[0024] In a seventh aspect, embodiments of this specification provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the above-described method steps.

[0025] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0026] The liveness detection model training method provided in the embodiments of this specification is adopted by constructing a training image dataset composed of training image data with multimodal images, and training a converged liveness detection model based on each training image data in the training image dataset. The training image data with multimodal face images ensures the detection accuracy of the liveness detection model. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating a liveness detection model training method provided in the embodiments of this specification;

[0029] Figure 2 A model structure diagram of a liveness detection model provided in the embodiments of this specification;

[0030] Figure 3 A model structure diagram of a liveness detection model provided in the embodiments of this specification;

[0031] Figure 4 A flowchart illustrating a liveness detection model training method provided in the embodiments of this specification;

[0032] Figure 5 A flowchart illustrating a liveness detection model training method provided in the embodiments of this specification;

[0033] Figure 6 A flowchart illustrating a liveness detection model training method provided in the embodiments of this specification;

[0034] Figure 7 A schematic flowchart of a liveness detection method provided in the embodiments of this specification;

[0035] Figure 8 This is a schematic diagram of the structure of a liveness detection model training device provided in the embodiments of this specification;

[0036] Figure 9 This is a schematic diagram of the structure of a liveness detection model training device provided in the embodiments of this specification;

[0037] Figure 10 This is a schematic diagram of the structure of a liveness detection device provided in the embodiments of this specification;

[0038] Figure 11 This is a structural block diagram of an electronic device provided as an embodiment of this specification. Detailed Implementation

[0039] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0040] In the description of this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this specification based on the specific circumstances. Furthermore, in the description of this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0041] In related technologies, with the continuous development of facial recognition systems, "liveness attack detection" has become an indispensable part of these systems, effectively intercepting non-liveness attack samples. As facial recognition technology is increasingly widely applied across various fields, some advanced attack methods have emerged, such as liveness injection attacks. These attacks bypass the camera and inject malicious video into the facial recognition system, thereby deceiving the online system. Therefore, how to effectively detect liveness injection attacks in facial recognition systems has become a pressing issue.

[0042] Based on this, the embodiments of this specification propose a liveness detection model training method. By constructing a training image dataset composed of training image data with multimodal images, and training a convergent liveness detection model based on each training image data in the training image dataset, the training image data with multimodal face images ensures the detection accuracy of the liveness detection model. Moreover, the liveness detection model trained based on multimodal images can effectively identify liveness injection attacks in the face recognition system and achieve high recall for various attack types faced by the face recognition system.

[0043] The following detailed description is provided in conjunction with embodiments of the examples in this specification. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims. The flowcharts shown in the accompanying drawings are merely illustrative and are not necessarily to be performed in accordance with the steps shown. For example, some steps are parallel and do not have a strict logical order; therefore, the actual execution order is variable.

[0044] Please see Figure 1 This is a flowchart illustrating a liveness detection model training method provided in an embodiment of this specification. In the embodiments of this specification, the liveness detection model training method is applied to a liveness detection model training device or an electronic device equipped with a liveness detection model training device. The following will focus on... Figure 1 The process shown will be described in detail. The liveness detection model training method may specifically include the following steps:

[0045] S102, Construct a training image dataset. Each training image in the training image dataset includes at least two modal images and a standard result on whether the modal image is a live organism.

[0046] In one embodiment of this specification, the training image data is sample face image data used to train a liveness detection model, which includes sample face images of at least two modalities and standard results of whether the sample face images are live.

[0047] The modal images include, but are not limited to, infrared images, depth images, and RGB images.

[0048] It should be noted that the standard results are used to characterize whether the training image data is based on real live images captured by a camera or on fake face images obtained through photos, videos, models, masks, etc. The training image dataset includes both training image data consisting of real live images and training image data consisting of fake face images.

[0049] S104, Input each modal image from the training image data into the liveness detection model, and use a feature extraction network to perform feature extraction processing on each modal image to obtain the image features corresponding to each modal image;

[0050] In one embodiment of this specification, the liveness detection model includes a feature extraction network, a feature fusion network, and a feature prediction network. After constructing a training image dataset, the training image data is input into the initial liveness detection model. The feature extraction network in the liveness detection model performs feature extraction processing on each modality image in the training image data to obtain the image features corresponding to each modality image.

[0051] Please see Figure 2 This is a model structure diagram of a liveness detection model provided in the embodiments of this specification. Figure 2 As shown, the liveness detection model includes a feature extraction network, a feature fusion network, and a feature prediction network.

[0052] In one embodiment of this specification, the feature extraction network of the liveness detection model can be multiple, and feature extraction processing is performed on each modality image corresponding to a feature extraction network to obtain the image features corresponding to each modality image.

[0053] It is understandable that by setting up independent feature extraction networks for different modal images and performing feature extraction processing on each modal image, and by not sharing network parameters between the feature extraction networks, the feature extraction effect for each modal image can be guaranteed.

[0054] Please see Figure 3 This is a model structure diagram of a liveness detection model provided in the embodiments of this specification. Figure 3 As shown, the liveness detection model includes multiple feature extraction networks, a feature fusion network, and a feature prediction network. For at least two modalities of images, independent feature extraction networks are used for feature extraction processing. The network parameters of each feature extraction network are not shared, and the independent feature extraction networks ensure the feature extraction effect for each modality of image.

[0055] In one embodiment of this specification, the liveness detection model further includes an attention network. After the feature extraction network performs feature extraction processing on each modality image to obtain the image features corresponding to each modality image, the attention mechanism of the attention network is used to perform feature enhancement processing on the image features corresponding to each modality image to obtain the enhanced image features.

[0056] It is understandable that attention mechanisms can be used to enhance the features of sensitive regions in modal images, and the enhanced image features can improve the accuracy of liveness detection models.

[0057] In one embodiment of this specification, the liveness detection model further includes a self-attention network. After the feature extraction network performs feature extraction processing on each modality image to obtain the image features corresponding to each modality image, the self-attention mechanism of the self-attention network is used to enhance the image features corresponding to each modality image. Based on the self-attention mechanism, the long-range relationship between features in different parts of the image features of the modality image can be established to obtain the image features after establishing the long-range relationship.

[0058] S106, A feature fusion network is used to perform feature fusion processing on each image feature to obtain fused image features;

[0059] In the embodiments of this specification, after the feature extraction network performs feature extraction processing on each modal image to obtain each image feature, the feature fusion network in the liveness detection model performs feature fusion processing on the image features corresponding to each modal image to obtain the fused image features of each image feature.

[0060] Optionally, in one embodiment of this specification, the liveness detection model further includes a feature enhancement network. After obtaining the fused image features, the feature enhancement network is used to enhance the fused image features to obtain the enhanced fused image features.

[0061] S108, a feature prediction network is used to perform liveness prediction on the fused image features to obtain the liveness prediction results corresponding to the training image data;

[0062] In the embodiments of this specification, after obtaining the fused image features of each modality image, the liveness prediction of the fused image features is performed based on the feature prediction network in the liveness prediction model to obtain the liveness prediction result corresponding to the training image data.

[0063] It should be noted that the liveness prediction result can be either "detected as live" or "detected as not live". The liveness prediction result is not correlated with the standard result corresponding to the training image data.

[0064] S110: Based on the preset loss function, the standard results corresponding to the training image data and the liveness prediction results, supervise the training of the liveness detection model and iteratively update the model parameters of the liveness detection model until the liveness detection model converges, and obtain the trained liveness detection model.

[0065] In the embodiments of this specification, after obtaining the liveness prediction result corresponding to the training image data, the loss value between the liveness prediction result and the standard result is calculated based on a preset loss function. The model parameters of the liveness detection model are updated and optimized based on the loss value. It is then determined whether the liveness detection model after updating and optimizing the model parameters has converged. If the model converges, training is stopped, and the trained liveness detection model is obtained. If the model does not converge, step S104 is executed to continue training the liveness detection model based on the training image data in the training image dataset until the model converges.

[0066] In one embodiment of this specification, the preset loss function includes a cross-entropy loss function and a prediction loss function set for each modality image. After obtaining the liveness prediction result corresponding to the training image data, the cross-entropy loss corresponding to each modality image is calculated according to the cross-entropy loss function corresponding to each modality image, and the model loss between the liveness prediction result and the standard result is calculated according to the prediction loss function. The network parameter values ​​of the feature extraction network corresponding to each modality image are updated and optimized based on the cross-entropy loss, and the network parameter values ​​of each part of the liveness detection model are updated and optimized based on the model loss.

[0067] It should be noted that the cross-entropy loss function is a loss function set for the feature extraction network. After the feature extraction network performs feature extraction processing on the modal image to obtain the image features corresponding to the modal image, the cross-entropy loss generated by the feature extraction network when extracting features from the modal image is calculated based on the liveness prediction results of the training image data and the standard results of the liveness detection model. The network parameters of the feature extraction network are then optimized based on the cross-entropy loss.

[0068] Optionally, in one embodiment of this specification, each modal image in at least two modal images corresponds to a feature extraction network. After calculating the cross-entropy loss corresponding to each modal image based on the cross-entropy loss function set for each modal image, the network parameters of the feature extraction network corresponding to each modal image are optimized based on the cross-entropy loss.

[0069] In the embodiments of this specification, a training image dataset consisting of training image data with multimodal images is constructed, and a convergent liveness detection model is generated based on each training image data in the training image dataset. The training image data with multimodal face images ensures the detection accuracy of the trained liveness detection model, and the liveness detection model trained based on multimodal images can effectively identify liveness injection attacks in the face recognition system and achieve high recall for various attack types faced by the face recognition system.

[0070] Please see Figure 4This is a flowchart illustrating a liveness detection model training method provided in an embodiment of this specification. The liveness detection model training method may include the following steps:

[0071] S202, Construct a training image dataset. Each training image in the training image dataset includes at least two modal images and a standard result on whether the modal image is a live organism.

[0072] In the embodiments of this specification, step S202 is described in detail in another embodiment of this specification for step S102, and will not be repeated here.

[0073] S204, Input each modal image from the training image data into the liveness detection model, and use a feature extraction network to perform feature extraction processing on each modal image to obtain the image features corresponding to each modal image;

[0074] In the embodiments of this specification, step S204 is described in detail in another embodiment of this specification for step S104, and will not be repeated here.

[0075] S206, A feature fusion network is used to perform feature fusion processing on each image feature to obtain fused image features;

[0076] In the embodiments of this specification, step S206 is described in detail in another embodiment of this specification for step S106, and will not be repeated here.

[0077] S208, a feature prediction network is used to perform liveness prediction on the fused image features to obtain the liveness prediction results corresponding to the training image data;

[0078] In the embodiments of this specification, step S208 is described in detail in another embodiment of this specification for step S108, and will not be repeated here.

[0079] S210, calculate the cross-entropy loss for each modality image based on the cross-entropy loss function, the standard results corresponding to the training image data, and the liveness prediction results;

[0080] In the embodiments of this specification, a corresponding cross-entropy loss function is preset for the feature extraction network corresponding to each modality image. This function is mainly used to optimize the network parameters of the feature extraction network based on the liveness prediction results of the training image data. After obtaining the liveness prediction results corresponding to the training image data through the feature prediction network, the cross-entropy loss corresponding to each modality image is calculated based on the preset cross-entropy loss function, the standard results corresponding to the training image data, and the liveness prediction results.

[0081] Optionally, in one embodiment of this specification, after obtaining the image features corresponding to each modality image, the method further includes: using a feature prediction network to perform liveness prediction on the image features corresponding to each modality image to obtain single-modality liveness prediction results corresponding to each modality image; calculating the cross-entropy loss corresponding to each modality image using the cross-entropy loss function corresponding to each modality image based on the single-modality liveness prediction results corresponding to each modality image and the standard results corresponding to the training image data; and updating and optimizing the network parameters of the corresponding feature extraction network based on the cross-entropy loss corresponding to each modality image.

[0082] S212, Calculate the prediction loss based on the prediction loss function, the standard results corresponding to the training image data, and the liveness prediction results;

[0083] In the embodiments of this specification, the prediction loss function is a loss function based on the global model settings, which is used to calculate the prediction loss based on the standard results and liveness prediction results corresponding to the training image data, and to adjust the network parameters of each part of the liveness detection model based on the prediction loss.

[0084] S214, Update the model parameters of the liveness detection model based on each cross-entropy loss and prediction loss;

[0085] In the embodiments of this specification, after calculating the cross-entropy loss corresponding to each modality image and the prediction loss corresponding to the liveness detection model, the network parameters of the feature extraction network corresponding to each modality image are updated and optimized based on the cross-entropy loss corresponding to each modality image, and the network parameters of each network in the liveness detection model are updated and optimized based on the prediction loss.

[0086] It is understood that in the embodiments of this specification, by setting a cross-entropy loss function for each modal image and calculating the cross-entropy loss of the feature extraction network when extracting features from the modal images, the feature extraction network is independently optimized based on the cross-entropy loss during the training process, thereby ensuring the feature extraction accuracy of the feature extraction network for each modal image and thus ensuring the detection accuracy of the liveness detection model.

[0087] Optionally, when the feature extraction network for each modality image is the same feature extraction network with shared network parameters, such as... Figure 2 The liveness detection model shown uses the same feature extraction network to extract features from each modality image. During training, the feature extraction network is updated and optimized based on the cross-entropy loss corresponding to each modality image.

[0088] Optionally, when the feature extraction networks for each modality image are independent feature extraction networks, such as... Figure 3The liveness detection model structure shown has independent feature extraction networks for each modality image. During training, the feature extraction networks for each modality image are updated and optimized based on the cross-entropy loss corresponding to each modality image, thus fully ensuring the independence of each feature extraction network.

[0089] S216, determine whether the liveness detection model with updated parameters meets the preset convergence condition. If it does, stop training and obtain the trained liveness detection model. If it does not meet the condition, proceed to step S204.

[0090] In the embodiments of this specification, a training image dataset consisting of training image data with multimodal images is constructed, and a convergent liveness detection model is generated based on each training image data in the training image dataset. The training image data with multimodal face images ensures the detection accuracy of the trained liveness detection model, and the liveness detection model trained based on multimodal images can effectively identify liveness injection attacks in the face recognition system and achieve high recall for various attack types faced by the face recognition system. During the training process, a cross-entropy loss function is set for each modality image to optimize the feature extraction network. The feature extraction network is independently optimized based on each cross-entropy loss function, which ensures the feature extraction network's accuracy in extracting features from each modality image, further ensuring the detection accuracy of the liveness detection model.

[0091] Please see Figure 5 This is a flowchart illustrating a liveness detection model training method provided in an embodiment of this specification. The liveness detection model training method may include the following steps:

[0092] S302, Construct a training image dataset. Each training image in the training image dataset includes at least two modal images and a standard result on whether the modal image is a live organism.

[0093] In the embodiments of this specification, step S302 is described in detail in another embodiment of this specification for step S102, and will not be repeated here.

[0094] S304, Input each modal image from the training image data into the liveness detection model, and use a feature extraction network to perform feature extraction processing on each modal image to obtain the image features corresponding to each modal image;

[0095] In the embodiments of this specification, step S304 is described in detail in another embodiment of this specification for step S104, and will not be repeated here.

[0096] S306, A feature fusion network is used to perform feature fusion processing on each image feature to obtain fused image features;

[0097] In the embodiments of this specification, for step S306, please refer to the detailed description of step S106 in another embodiment of this specification, which will not be repeated here.

[0098] S308, a feature prediction network is used to perform liveness prediction on the fused image features to obtain the liveness prediction result corresponding to the training image data;

[0099] In the embodiments of this specification, step S308 is described in detail in another embodiment of this specification for step S108, and will not be repeated here.

[0100] S310, calculate the cross-entropy loss for each modality image based on each cross-entropy loss function, the standard results corresponding to the training image data, and the liveness prediction results;

[0101] In the embodiments of this specification, step S310 is described in detail in another embodiment of this specification for step S210, and will not be repeated here.

[0102] S312, calculate the uncertainty corresponding to each cross-entropy loss based on the uncertainty-weighted loss function;

[0103] In the embodiments of this specification, an uncertainty-weighted loss function is set for each cross-entropy loss function. After the cross-entropy loss is calculated based on each cross-entropy loss function, the uncertainty-weighted loss function can balance each cross-entropy loss according to the random uncertainty of each cross-entropy loss, so as to realize the adaptive cross-entropy loss weight during training. Finally, the weight of the cross-entropy loss corresponding to each modality image reaches the optimal configuration, thereby improving the prediction accuracy of the model.

[0104] S314, Calculate the prediction loss based on the prediction loss function, the standard results corresponding to the training image data, and the liveness prediction results;

[0105] In the embodiments of this specification, the prediction loss function is a loss function based on the global model settings, which is used to calculate the prediction loss based on the standard results and liveness prediction results corresponding to the training image data, and to adjust the network parameters of each part of the liveness detection model based on the prediction loss.

[0106] S316, update the model parameters of the liveness detection model based on each cross-entropy loss, the uncertainty corresponding to each cross-entropy loss, and the prediction loss;

[0107] In the embodiments of this specification, the uncertainty corresponding to each cross-entropy loss is used to characterize the weight corresponding to each cross-entropy loss. The greater the uncertainty, the lower the corresponding weight. The network parameters of the feature extraction network in the liveness detection model are updated and optimized based on each cross-entropy loss and the uncertainty corresponding to each cross-entropy loss. The model parameters of each network in the liveness detection model are updated based on the prediction loss.

[0108] In one embodiment of this specification, the modal images in the training image data may include infrared images, depth images, and RGB images. The feature extraction network in the liveness detection model includes three feature extraction networks corresponding to the infrared image, depth image, and RGB image, respectively. After inputting the infrared image, depth image, and RGB image from the same training image data into the liveness detection model, the feature extraction networks corresponding to the infrared image, depth image, and RGB image respectively perform feature extraction processing on each image to obtain image features corresponding to the infrared image, depth image, and RGB image, respectively. Then, a feature fusion network fuses the image features to obtain fused image features. Finally, a feature prediction network performs liveness prediction on the fused image features to obtain the liveness prediction result. Then, based on the liveness prediction result, the standard result corresponding to the training image data, and the cross-entropy loss function set for each modality image, the cross-entropy loss corresponding to each modality image is calculated. The prediction loss is calculated based on the prediction loss function, the standard result corresponding to the training image data, and the liveness prediction result. The uncertainty corresponding to each cross-entropy loss is calculated based on the uncertainty-weighted loss function. The model parameters of the liveness detection model are updated based on each cross-entropy loss, the uncertainty corresponding to each cross-entropy loss, and the prediction loss. Specifically, the network parameters of each feature extraction network are updated and optimized based on the cross-entropy loss and the uncertainty corresponding to each cross-entropy loss.

[0109] Optionally, when calculating the uncertainty corresponding to each cross-entropy loss according to the uncertainty weighted loss function, the uncertainty can be calculated as follows: calculate the uncertainty of the cross-entropy loss corresponding to the depth image and the infrared image respectively according to the uncertainty weighted loss function, and then update the model parameters of the liveness detection model based on the uncertainty of the cross-entropy loss corresponding to the RGB image, the cross-entropy loss corresponding to the depth image, the cross-entropy loss corresponding to the infrared image, the cross-entropy loss corresponding to the depth image and the infrared image respectively, and the prediction loss.

[0110] S318, determine whether the liveness detection model with updated parameters meets the preset convergence condition. If it does, stop training and obtain the trained liveness detection model. If it does not meet the condition, proceed to step S304.

[0111] In the embodiments of this specification, a training image dataset consisting of training image data with multimodal images is constructed, and a convergent liveness detection model is generated based on each training image data in the training image dataset. The training image data with multimodal face images ensures the detection accuracy of the trained liveness detection model, and the liveness detection model trained based on multimodal images can effectively identify liveness injection attacks in the face recognition system and achieve high recall for various attack types faced by the face recognition system. During the training process, a cross-entropy loss function is set for each modality image to optimize the feature extraction network. The feature extraction network is independently optimized based on each cross-entropy loss function, ensuring the feature extraction network's accuracy in extracting features from each modality image, further ensuring the detection accuracy of the liveness detection model. Furthermore, based on the uncertainty-weighted loss function, the cross-entropy loss is balanced according to the random uncertainty of each cross-entropy loss, realizing the adaptive cross-entropy loss weight during the training process. Finally, the weights of the cross-entropy loss corresponding to each modality image are optimally configured, further improving the prediction accuracy of the model.

[0112] Please see Figure 6 This is a flowchart illustrating a liveness detection model training method provided in an embodiment of this specification. The liveness detection model training method may include the following steps:

[0113] S402, Construct a training image dataset. Each training image in the training image dataset includes at least two modal images and a standard result on whether the modal image is a live organism.

[0114] In the embodiments of this specification, step S402 is described in detail in another embodiment of this specification for step S102, and will not be repeated here.

[0115] S404, Input each modal image from the training image data into the liveness detection model, and use a feature extraction network to perform feature extraction processing on each modal image to obtain the image features corresponding to each modal image;

[0116] In the embodiments of this specification, step S404 is described in detail in another embodiment of this specification for step S104, and will not be repeated here.

[0117] S406, Based on the attention network, feature enhancement processing is performed on each image feature to obtain the enhanced image features;

[0118] It is understandable that attention mechanisms can be used to enhance the features of sensitive regions in modal images, and the enhanced image features can improve the accuracy of liveness detection models.

[0119] S408, Based on a self-attention network, establish the long-range relationship between features of each part of the image features to obtain the image features after establishing the long-range relationship;

[0120] It is understandable that there is some kind of connection between different parts of a modal image. After extracting the image features corresponding to the modal image, the self-attention network is used to establish the long-range relationship between the image features of different regions of the modal image, making the image features more distinct and helping to improve the prediction accuracy of the liveness detection model.

[0121] Optionally, during training, the network parameters of the feature extraction network, attention network, and self-attention network are updated and optimized based on the cross-entropy loss of each modality image.

[0122] S410, a feature fusion network is used to perform feature fusion processing on each image feature to obtain fused image features;

[0123] In the embodiments of this specification, step S410 is described in detail in another embodiment of this specification for step S106, and will not be repeated here.

[0124] S412, Based on the feature enhancement network, feature enhancement processing is performed on the fused image features to obtain the enhanced fused image features;

[0125] S414, A feature prediction network is used to perform liveness prediction on the fused image features to obtain the liveness prediction result corresponding to the training image data;

[0126] In the embodiments of this specification, step S414 is described in detail in another embodiment of this specification for step S108, and will not be repeated here.

[0127] S416. Based on the preset loss function, the standard results corresponding to the training image data and the liveness prediction results, the liveness detection model is trained under supervision and the model parameters of the liveness detection model are iteratively updated until the liveness detection model converges, and the trained liveness detection model is obtained.

[0128] In the embodiments of this specification, step S416 is described in detail in another embodiment of this specification for step S110, and will not be repeated here.

[0129] In the embodiments of this specification, a training image dataset consisting of training image data with multimodal images is constructed, and a convergent liveness detection model is generated based on each training image data in the training image dataset. The training image data with multimodal face images ensures the detection accuracy of the trained liveness detection model, and the liveness detection model trained based on multimodal images can effectively identify liveness injection attacks in the face recognition system and achieve high recall for various attack types faced by the face recognition system.

[0130] Please see Figure 7 This is a schematic flowchart illustrating a liveness detection method provided in an embodiment of this specification. In the embodiments of this specification, the liveness detection method is applied to a liveness detection device or an electronic device equipped with a liveness detection device. The following will focus on... Figure 1 The process shown will be described in detail. The liveness detection method may specifically include the following steps:

[0131] S502, acquire at least two modal images of the face to be detected;

[0132] In one feasible implementation, at least two modal images of the face to be detected can be acquired based on an image acquisition device.

[0133] In one embodiment of this specification, the at least two modal images may include infrared images, depth images, and RGB images. An infrared image of the face to be detected is acquired using an infrared image acquisition device, a depth image of the face to be detected is acquired using a depth image acquisition device, and an RGB image of the face to be detected is acquired using a high-definition camera device.

[0134] S504, input at least two modal images into the trained liveness detection model obtained by the liveness detection model training method described above, and output the liveness detection result.

[0135] In the embodiments of this specification, at least two modal images corresponding to the face to be detected are input into a liveness detection model that has been trained using the liveness detection model training method described above, and the liveness detection model outputs the liveness detection results for the at least two modal images.

[0136] Liveness detection results are used to characterize whether the face to be detected is a live person corresponding to at least two modalities of images.

[0137] In the embodiments of this specification, multimodal face images of the face to be detected are acquired, and then the liveness detection model trained in the above embodiments is used to detect whether the multimodal face images of the face to be detected are live. The liveness detection model trained on multimodal images can effectively identify liveness injection attacks in the face recognition system and achieve high recall for various attack types faced by the face recognition system, effectively improving the reliability and security of the face recognition system.

[0138] Please see Figure 8 This is a schematic diagram of the structure of a liveness detection model training device provided in an embodiment of this specification. Figure 8 As shown, the liveness detection model training device 1 can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the liveness detection model training device 1 includes a dataset construction module 11, a feature extraction module 12, a feature fusion module 13, a liveness prediction module 14, and a parameter update module 15, specifically including:

[0139] The dataset construction module 11 is used to construct a training image dataset, wherein each training image data in the training image dataset includes at least two modal images and a standard result of whether the modal image is a live organism;

[0140] Feature extraction module 12 is used to input each modal image in the training image data into the liveness detection model, and use the feature extraction network to perform feature extraction processing on each modal image to obtain the image features corresponding to each modal image.

[0141] Feature fusion module 13 is used to perform feature fusion processing on each of the image features using the feature fusion network to obtain fused image features;

[0142] The liveness prediction module 14 is used to perform liveness prediction on the fused image features using the feature prediction network to obtain the liveness prediction result corresponding to the training image data.

[0143] The parameter update module 15 is used to supervise the training of the liveness detection model based on a preset loss function, the standard results corresponding to the training image data, and the liveness prediction results, and to iteratively update the model parameters of the liveness detection model until the liveness detection model converges, thereby obtaining the trained liveness detection model.

[0144] Optionally, the preset loss function includes a cross-entropy loss function and a prediction loss function set for each of the modal images respectively, and the parameter update module 15 is specifically used for:

[0145] The cross-entropy loss for each modality image is calculated based on the cross-entropy loss function, the standard result corresponding to the training image data, and the liveness prediction result.

[0146] The prediction loss is calculated based on the prediction loss function, the standard result corresponding to the training image data, and the liveness prediction result.

[0147] The model parameters of the liveness detection model are updated based on the cross-entropy loss and the prediction loss.

[0148] Determine whether the parameter-updated liveness detection model meets the preset convergence condition. If it does, stop training and obtain the trained liveness detection model. If it does not meet the condition, proceed with the steps of inputting each modality image from the training image data into the liveness detection model and using the feature extraction network to perform feature extraction processing on each modality image to obtain the image features corresponding to each modality image.

[0149] Optionally, the preset loss function further includes an uncertainty-weighted loss function set for each of the cross-entropy loss functions. After the parameter update module 15 performs the calculation of the cross-entropy loss for each modality image based on each of the cross-entropy loss functions, the standard result corresponding to the training image data, and the liveness prediction result, it is further configured to:

[0150] The uncertainty corresponding to each cross-entropy loss is calculated based on the uncertainty-weighted loss function.

[0151] The model parameters of the liveness detection model are updated based on the cross-entropy loss, the uncertainty corresponding to the cross-entropy loss, and the prediction loss.

[0152] Optional, please see Figure 9 This is a schematic diagram of the structure of a liveness detection model training device provided in an embodiment of this specification. Figure 9 As shown, the liveness detection model training device further includes a first feature enhancement module 16, used for:

[0153] Based on the attention network, feature enhancement processing is performed on each of the image features to obtain the enhanced image features.

[0154] Optional, such as Figure 9 As shown, the liveness detection model training device further includes a second feature enhancement module 17, used for:

[0155] Based on the self-attention network, the long-range relationships between the features of each part of the image feature are established to obtain the image feature after establishing the long-range relationships.

[0156] Optional, such as Figure 9 As shown, the liveness detection model training device further includes a third feature enhancement module 18, used for:

[0157] The fused image features are enhanced based on the feature enhancement network to obtain the enhanced fused image features.

[0158] In the embodiments of this specification, a training image dataset consisting of training image data with multimodal images is constructed, and a convergent liveness detection model is generated based on each training image data in the training image dataset. The training image data with multimodal face images ensures the detection accuracy of the trained liveness detection model, and the liveness detection model trained based on multimodal images can effectively identify liveness injection attacks in the face recognition system and achieve high recall for various attack types faced by the face recognition system.

[0159] Please see Figure 10 This is a schematic diagram of the structure of a liveness detection device provided in an embodiment of this specification. Figure 10 As shown, the liveness detection device 2 can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the liveness detection device 2 includes an image acquisition module 21 and a liveness detection module 22, specifically including:

[0160] Image acquisition module 21 is used to acquire at least two modal images of the face to be detected;

[0161] The liveness detection module 22 is used to input the at least two modal images into the trained liveness detection model and output the liveness detection result.

[0162] In the embodiments of this specification, multimodal face images of the face to be detected are acquired, and then the liveness detection model trained in the above embodiments is used to detect whether the multimodal face images of the face to be detected are live. The liveness detection model trained on multimodal images can effectively identify liveness injection attacks in the face recognition system and achieve high recall for various attack types faced by the face recognition system, effectively improving the reliability and security of the face recognition system.

[0163] It should be noted that the mouth animation generation device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the mouth animation generation method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the mouth animation generation device and the mouth animation generation method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0164] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0165] This specification also provides an embodiment of a computer storage medium that can store multiple instructions adapted to be loaded and executed by a processor as described above. Figures 1 to 7 The liveness detection model training method described in the illustrated embodiment can be found in the following documentation for a detailed execution process: Figures 1 to 7 The specific details of the illustrated embodiments will not be elaborated here.

[0166] This specification also provides a computer program product that stores at least one instruction, said at least one instruction being loaded and executed by the processor as described above. Figures 1 to 7 The liveness detection model training method described in the illustrated embodiment can be found in the following documentation for a detailed execution process: Figures 1 to 7 The specific details of the illustrated embodiments will not be elaborated here.

[0167] Please refer to Figure 11 This is a structural block diagram of an electronic device provided in an embodiment of this specification. The electronic device in this specification may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 can be connected via the bus 150.

[0168] Processor 110 may include one or more processing cores. Processor 110 connects to various parts of the terminal using various interfaces and lines, and performs various functions and processes data of terminal 100 by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 110 may integrate one or more of the following: central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 110 and may be implemented separately using a communication chip.

[0169] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 may include non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets.

[0170] The input device 130 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 140 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In this embodiment, the input device 130 can be a temperature sensor to obtain the operating temperature of the terminal. The output device 140 can be a speaker to output audio signals.

[0171] In addition, those skilled in the art will understand that the structure of the terminal shown in the above figures does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the terminal may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WIFI) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0172] In the embodiments of this specification, the executing entity for each step can be the terminal described above. Optionally, the executing entity for each step is the terminal's operating system. The operating system can be Android, iOS, or other operating systems; this specification does not limit this.

[0173] exist Figure 11 In the electronic device, the processor 110 can be used to call the liveness detection model training program stored in the memory 120 and execute it to implement the liveness detection model training method as described in the various method embodiments of this specification.

[0174] In the embodiments of this specification, a training image dataset consisting of training image data with multimodal images is constructed, and a convergent liveness detection model is generated based on each training image data in the training image dataset. The training image data with multimodal face images ensures the detection accuracy of the trained liveness detection model, and the liveness detection model trained based on multimodal images can effectively identify liveness injection attacks in the face recognition system and achieve high recall for various attack types faced by the face recognition system.

[0175] Those skilled in the art will clearly understand that the technical solutions in this specification can be implemented using software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware capable of independently performing or cooperating with other components to perform a specific function. Hardware may include, for example, a Field-Programmable Gate Array (FPGA), an Integrated Circuit (IC), etc.

[0176] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this specification is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this specification. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this specification.

[0177] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0178] In the several embodiments provided in this specification, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0179] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0180] Furthermore, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0181] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0182] The foregoing descriptions are merely exemplary embodiments of this specification and should not be construed as limiting the scope of this specification. Any equivalent changes and modifications made in accordance with the teachings of this specification shall still fall within the scope of this specification. Other embodiments of this specification will be readily apparent to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this specification are defined by the claims.

Claims

1. A method for training a liveness detection model, wherein the liveness detection model includes a feature extraction network, a feature fusion network, and a feature prediction network, the method comprising: Construct a training image dataset, wherein each training image in the training image dataset includes at least two modalities of images and a standard result of whether the modality of the image is a live organism; Each modal image in the training image data is input into the liveness detection model, and the feature extraction network is used to perform feature extraction processing on each modal image to obtain the image features corresponding to each modal image. The feature fusion network is used to perform feature fusion processing on each of the image features to obtain fused image features; The feature prediction network is used to perform liveness prediction on the fused image features to obtain the liveness prediction result corresponding to the training image data; The cross-entropy loss for each modality image is calculated based on each cross-entropy loss function, the standard result corresponding to the training image data, and the liveness prediction result. The uncertainty corresponding to each cross-entropy loss is calculated based on the uncertainty-weighted loss function. The prediction loss is calculated based on a preset loss function, the standard result corresponding to the training image data, and the liveness prediction result. The preset loss function includes a cross-entropy loss function and a prediction loss function set for each modality image. The preset loss function also includes an uncertainty-weighted loss function set for each cross-entropy loss function. The model parameters of the liveness detection model are updated based on the cross-entropy loss, the uncertainty corresponding to the cross-entropy loss, and the prediction loss.

2. The method according to claim 1, further comprising: Determine whether the parameter-updated liveness detection model meets the preset convergence condition. If it does, stop training and obtain the trained liveness detection model. If it does not meet the condition, proceed with the steps of inputting each modality image from the training image data into the liveness detection model and using the feature extraction network to perform feature extraction processing on each modality image to obtain the image features corresponding to each modality image.

3. The method according to claim 1, wherein the liveness detection model further comprises an attention network; Before performing feature fusion processing on each of the image features using the feature fusion network to obtain the fused image features, the method further includes: Based on the attention network, feature enhancement processing is performed on each of the image features to obtain the enhanced image features.

4. The method according to claim 1, wherein the liveness detection model further comprises a self-attention network; Before performing feature fusion processing on each of the image features using the feature fusion network to obtain the fused image features, the method further includes: Based on the self-attention network, the long-range relationships between the features of each part of the image feature are established to obtain the image feature after establishing the long-range relationships.

5. The method according to claim 1, wherein the liveness detection model further comprises a feature enhancement network; Before using the feature prediction network to perform liveness prediction on the fused image features to obtain the liveness prediction result corresponding to the training image data, the method further includes: The fused image features are enhanced based on the feature enhancement network to obtain the enhanced fused image features.

6. A method for detecting liveness, the method comprising: Acquire at least two modal images of the face to be detected; The at least two modal images are input into a liveness detection model that has been trained using the liveness detection model training method as described in any one of claims 1 to 5, and the liveness detection result is output.

7. A liveness detection model training device, comprising: The dataset construction module is used to construct a training image dataset, wherein each training image in the training image dataset includes at least two modal images and a standard result of whether the modal image is a live organism; The feature extraction module is used to input each modal image in the training image data into the liveness detection model, and to use a feature extraction network to perform feature extraction processing on each modal image to obtain the image features corresponding to each modal image. The feature fusion module is used to perform feature fusion processing on the image features using a feature fusion network to obtain fused image features; The liveness prediction module is used to perform liveness prediction on the fused image features using a feature prediction network to obtain the liveness prediction result corresponding to the training image data. The parameter update module is used to calculate the cross-entropy loss for each modality image based on each cross-entropy loss function, the standard result corresponding to the training image data, and the liveness prediction result; calculate the uncertainty corresponding to each cross-entropy loss based on the uncertainty-weighted loss function; calculate the prediction loss based on the preset loss function, the standard result corresponding to the training image data, and the liveness prediction result, wherein the preset loss function includes a cross-entropy loss function and a prediction loss function set for each modality image, and the preset loss function also includes an uncertainty-weighted loss function set for each cross-entropy loss function; and update the model parameters of the liveness detection model based on each cross-entropy loss, the uncertainty corresponding to each cross-entropy loss, and the prediction loss.

8. A liveness detection device, comprising: The image acquisition module is used to acquire at least two modal images of the face to be detected; The liveness detection module is used to input the at least two modal images into a trained liveness detection model obtained by the liveness detection model training method as described in any one of claims 1 to 5, and output liveness detection results.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as claimed in any one of claims 1 to 6.

11. A computer program product having at least one instruction stored thereon, characterized in that, When the at least one instruction is executed by the processor, it implements the steps of the method according to any one of claims 1 to 6.

Images