Construction method of face liveness detection and classification model, detection and classification method and device

Abstract

The present invention discloses a method for constructing a face liveness detection classification model, a detection classification method, and an apparatus, comprising obtaining training samples; constructing a deep neural network for face liveness detection; preprocessing the training samples to construct a first training set; for each sample in the first training set, generating a corresponding adversarial sample and adding it to the first training set to form a second training set; constructing a defensive training loss function for face liveness detection, and iteratively optimizing the deep neural network using the second training set to obtain a robust face liveness detection classification model. The classification method includes: obtaining raw test data and preprocessing it; inputting the test data into the above-mentioned face liveness detection classification model, and outputting a detection result. The method can effectively overcome the interference caused by adversarial samples on the face liveness detection classifier, thereby effectively improving the accuracy and reliability of identity authentication systems with face liveness detection functions.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method for constructing a face liveness detection and classification model, a detection and classification method, and a device. Background Art

[0002] With the development of biometric recognition technology, facial recognition has become widely used in access control systems, security inspection systems, and criminal investigation systems due to its high security, intuitiveness, and user-friendliness. However, the widespread use of the internet has made facial information easy to obtain and forge, posing a serious threat to systems that rely on facial biometric authentication. For example, an unauthorized user can gain access to a system by using a photo, video, or 3D model of a legitimate user. To ensure more reliable authentication, facial liveness detection technology is currently widely used to determine whether the biometric information obtained by the system originates from the legitimate user's body or a fake one.

[0003] In practical applications, commonly used face liveness detection methods include: those based on facial motion information, which require significant user cooperation, are sensitive to lighting, and have low reliability; those based on infrared imaging, which, while highly accurate, often require additional hardware; and those based on texture information, which are sensitive to noise. Furthermore, most of these algorithms rely on manual feature extraction, making it difficult to accurately describe facial features in complex environments. To address this issue, a number of face liveness detection methods based on deep neural networks have emerged in recent years.

[0004] However, deep neural networks have been proven to be vulnerable to adversarial sample attacks, which manifests itself in that by adding subtle perturbations that are imperceptible to the naked eye to the original samples, the deep neural network can output any preset classification results with extremely high confidence. In face recognition-based identity recognition, adversarial sample attacks specifically manifest themselves in that illegal users can gain access to legitimate users and invade the system by adding or attaching carefully designed perturbations to legitimate users' photos, videos, or three-dimensional molds. Similarly, recognition systems based on liveness detection also face adversarial sample attack problems, such as Figure 1 As shown. Among them, Figure 1 (a) in the figure is a face image of a living object. Figure 1 (b) in the figure is the secondary imaging of the prosthetic image, i.e., the conventional image attack. Figure 1 (c) in the figure is a carefully designed perturbation for conventional image attacks. Figure 1 (d) in Figure 1 (b) superposition Figure 1Conventional face liveness detection methods based on deep neural networks have almost no defense against adversarial examples. Illegal users can trick the liveness detection system into confidently identifying the prosthetic object as live by making subtle interference in the re-photographed image, thereby invading the recognition system.

[0005] The face liveness detection task has its own characteristics in terms of adversarial sample attacks. First, liveness detection is a binary classification problem, and the final output categories are live objects and prosthetic objects. Among them, the difference between live objects and prosthetic objects in the image is very small, so the inter-class distance between the images of live objects and prosthetic objects is already very small. In addition, adding adversarial samples to the face liveness detection task to interfere with the detection of live objects and prostheses greatly increases the difficulty of liveness detection and further increases the requirements for the discriminability of the liveness detection network. Drawing on the fact that metric learning under conventional training can effectively narrow the feature distance of similar samples and push away the feature distance of non-similar samples, the present invention integrates it with adversarial training to implement adversarial sample defense for face liveness detection. Summary of the Invention

[0006] The purpose of the embodiments of the present application is to provide a method for constructing a face liveness detection and classification model, a detection and classification method, and a device to solve the problem that existing face liveness detection technology lacks the ability to defend against adversarial samples, improve the reliability of face liveness detection devices based on deep neural networks, and thereby achieve the reliability of identity authentication systems with face liveness detection functions.

[0007] According to a first aspect of an embodiment of the present application, a method for constructing a face liveness detection classification model is provided, comprising:

[0008] Get training samples;

[0009] Build a deep neural network for face liveness detection;

[0010] Preprocessing the training samples to construct a first training set;

[0011] For each sample in the first training set, generate a corresponding adversarial sample and add it to the first training set to form a second training set;

[0012] A defensive training loss function for face liveness detection is constructed, and the second training set is used to iteratively optimize the deep neural network to obtain a robust face liveness detection classification model.

[0013] Furthermore, the adversarial example is formally represented as follows:

[0014]

[0015] in, represents the vector representation of the adversarial perturbation, , Indicates the imperceptibility of the disturbance, Represents the face image in the regular sample training set, which is the image of a living object or the secondary imaging of a prosthetic object. Represents a face image Superimpose adversarial disturbances Post-synthesized adversarial examples, Representing a deep neural network Adversarial examples A face image classified as a living object, that is, the current detected object is judged to be a living object.

[0016] Furthermore, the defensive training loss function L for face liveness detection is formally expressed as follows:

[0017] ,

[0018] in, represents the loss function for the classification problem, represents the adversarial metric learning loss function, represents the regularization term, , and are the weight coefficients of each loss term;

[0019] The classification loss function The formal representation is as follows:

[0020]

[0021] in, represents the binary classification loss function, a face image representing a living subject, x represents the secondary image of the prosthetic object, represents adversarial samples, Indicates that the category label is living, Indicates that the category label is prosthesis;

[0022] The adversarial metric learning loss function The formal representation is as follows:

[0023]

[0024] Where T represents the triplet loss function, g(·) represents the representation of the input image by the deep neural network, Represents the deep neural network representation of input image x1 and input image x2 and The distance between them, a is the preset boundary value;

[0025] The regularization term The formal representation is as follows:

[0026]

[0027] in, represents regularization with norm p.

[0028] According to a second aspect of an embodiment of the present application, a device for constructing a face liveness detection classification model is provided, comprising:

[0029] A first acquisition module is used to acquire training samples;

[0030] The first building block is used to build a deep neural network for face liveness detection;

[0031] A second construction module is used to preprocess the training samples to construct a first training set;

[0032] A third construction module is configured to generate a corresponding adversarial sample for each sample in the first training set and add the sample to the first training set to form a second training set;

[0033] A model building module is used to construct a defensive training loss function for face liveness detection, and use the second training set to iteratively optimize the deep neural network to obtain a robust face liveness detection classification model.

[0034] According to a third aspect of an embodiment of the present application, a method for face liveness detection and classification is provided, comprising:

[0035] Obtain the original test data and perform preprocessing;

[0036] Input the data to be tested into the above-mentioned face liveness detection classification model and output the detection results.

[0037] Furthermore, the data to be tested is obtained and preprocessed, including:

[0038] Determine the form of the data to be tested;

[0039] If the acquired test data is image data, then directly detect whether the image contains a human face. If so, preprocess the image; if not, do not process it. If the acquired test data is video data, extract a single-frame image from the video sequence as the test image, detect whether the test image contains a human face, and if so, preprocess the image. If not, continue to extract single-frame images from the video sequence to be tested, detect whether the single-frame image contains a human face, and repeat this process until a face is detected in the single-frame image, and preprocess the single-frame image.

[0040] Furthermore, the robust face liveness detection and classification model can not only distinguish between the image of a live object and the secondary imaging of a prosthetic object, but also correctly classify the adversarial sample as a prosthetic object, that is,

[0041] ,and ,and .

[0042] According to a fourth aspect of an embodiment of the present application, a face liveness detection and classification device is provided, comprising:

[0043] The second acquisition module is used to obtain the original test data and perform preprocessing;

[0044] The detection module is used to input the data to be tested into the face liveness detection classification model described in the first aspect and output the detection results.

[0045] According to a fifth aspect of the embodiments of the present application, there is provided an electronic device, including:

[0046] one or more processors;

[0047] a memory for storing one or more programs;

[0048] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect or the third aspect.

[0049] According to a sixth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored, characterized in that when the instructions are executed by a processor, the steps of the method described in the first aspect or the third aspect are implemented.

[0050] The technical solutions provided by the embodiments of the present application may have the following beneficial effects:

[0051] As can be seen from the above examples, this application, through the application of metric learning theory, effectively narrows the feature distances between similar samples (secondary re-images of both prosthetics and adversarial samples) and pushes the feature distances between dissimilar samples (images of live subjects and prosthetic subjects). This addresses the technical difficulty of distinguishing between live and prosthetic subjects in face liveness detection tasks, where the imaging differences are minimal and difficult to distinguish. Furthermore, this application incorporates the concept of adversarial training on the basis of conventional face liveness detection, effectively overcoming the interference caused by adversarial samples on face liveness detection classification models, thereby effectively improving the accuracy and reliability of identity authentication systems with face liveness detection capabilities.

[0052] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0054] Figure 1 Schematic diagram of adversarial sample generation in face liveness detection, where (a) is a face image of a living object, (b) is a secondary image of a prosthetic object (e.g., video, image), (c) is the adversarial perturbation, and (d) is the adversarial sample constructed by superimposing (b) and (c).

[0055] Figure 2 This is a flowchart of a method for constructing a face liveness detection classification model provided in Example 1 of the present invention.

[0056] Figure 3 This is a block diagram of a device for constructing a face liveness detection classification model provided in Example 2 of the present invention.

[0057] Figure 4 This is a flowchart of a face liveness detection and classification method provided in Example 3 of the present invention.

[0058] Figure 5 This is a block diagram of a face liveness detection and classification device provided in Example 4 of the present invention.

[0059] Figure 6 This is a block diagram of a face liveness detection and classification system provided in Example 5 of the present invention.

[0060] Figure 7 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0062] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0063] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0064] Example 1:

[0065] Please refer to Figure 2 , Figure 2 This is a flowchart of a method for constructing a face liveness detection classification model provided by Example 1 of the present invention. Figure 2 The process shown is described in detail, and the method includes:

[0066] Step S100: Obtain training samples.

[0067] Specifically, the training samples may include: a single-frame image extracted from a video sequence of a live legitimate user captured by a camera, or an image captured by an illegal user in a camera by bending or rotating a photo of the legitimate user, playing a video of the legitimate user, or presenting a three-dimensional face model of the legitimate user.

[0068] Specifically, the obtaining of training samples includes: obtaining training samples in video form, randomly extracting a single-frame image from a video sequence, detecting whether the single-frame image contains a face, if so, retaining the image and ending the random extraction of the single-frame image, if not, discarding the image, continuing to randomly extract the remaining single-frame images in the video sequence, detecting whether the single-frame image contains a face, and repeating this process until a face is detected in the single-frame image; or obtaining training samples in image form, detecting whether the image contains a face, if so, retaining the image, and if not, discarding the image.

[0069] Detecting whether an image contains a face can be done using a machine learning-based or skin color-based face detection method. Preferably, this embodiment uses a normalized pixel difference method, which has high accuracy and high speed. If a face image is detected, preprocessing is performed on the single frame. Otherwise, the current image is not processed and the process jumps to the next image for face detection.

[0070] Step S200: Construct a deep neural network for face liveness detection.

[0071] Specifically, the deep neural network used for face liveness detection is a binary classifier, which ultimately outputs two categories: living objects and prosthetic objects.

[0072] The construction of a deep neural network requires comprehensive consideration of multiple aspects such as performance indicators, the number of training samples, and available computing resources. Smaller networks train faster but have lower prediction accuracy. Conversely, larger networks converge more slowly but have better prediction accuracy. Preferably, this embodiment uses the deep neural network model ResNet18 as the backbone network to balance training efficiency and effectiveness. In view of the main purpose of the present invention, the number of output categories of the last fully connected layer is set to 2, specifically: living body, prosthesis.

[0073] Step S300: pre-processing the training samples to construct a first training set;

[0074] Specifically, the preprocessing includes: face positioning; image enhancement; and image normalization.

[0075] Preferably, based on the aforementioned face detection, a face positioning method FPS-LBF based on local binary features is used to locate the 68 key points of the face, such as the eyes, eyebrows, corners of the mouth and other contour points, so as to more finely cut out the facial image; considering that the facial images collected in the actual environment often contain more noise, this embodiment uses the Gaussian curvature filtering method to enhance the image, while reducing the image noise and effectively retaining the edge features and texture details; then, the facial image is scaled using bilinear interpolation so that the image size meets the model input requirements. In this embodiment, the model input requirement is 224*224. In other embodiments, the image scaling method can also be nearest neighbor interpolation, bilinear interpolation or pixel area interpolation. Finally, the pixel value x is normalized:

[0076]

[0077] in is the mean value of image pixels, is the image pixel standard deviation, is the normalized pixel value.

[0078] It can be understood that the above preprocessing is to remove background noise in the data and enhance signals that help recognition, thereby destroying the perturbation of adversarial samples to a certain extent.

[0079] Specifically, the first training set includes: facial images of living objects and secondary images of prosthetic objects.

[0080] As an embodiment, the first training set includes the facial image of the aforementioned living object, the secondary imaging of the prosthetic object and the corresponding category information of the image, namely: living and prosthetic, respectively representing the image collected from the action set of the user's living body, or the action set collected from the prosthetic object of the user.

[0081] Step S400: For each sample in the first training set, generate a corresponding adversarial sample and add it to the first training set to form a second training set.

[0082] Specifically, the adversarial example is formally represented as follows:

[0083]

[0084] in, represents the vector representation of the adversarial perturbation, , Indicates the imperceptibility of the disturbance,

[0085] represents the face image in the regular sample training set, which can be the image of a living object or the secondary imaging of a prosthetic object. Represents a face image Superimpose adversarial disturbances Post-synthesized adversarial examples, Representing a deep neural network Adversarial examples A face image classified as a living object, that is, the current detected object is judged to be a living object.

[0086] In actual implementation, a variety of different methods can be used to solve the above optimization problem, such as the Fast Gradient Sign Method (FGSM), methods based on Projected Gradient Descend (PGD), and Contrast Reduction Attack (CRD).

[0087] Preferably, this embodiment uses the PGD method to generate adversarial perturbations, and adopts p=2 to obtain adversarial samples of facial images with less perturbations and stronger aggressiveness.

[0088] Specifically, for each sample in the first training set, a corresponding adversarial sample is generated.

[0089] Preferably, for each living object sample and prosthetic image sample in the first training set, an adversarial sample generated by the aforementioned PGD method is used.

[0090] Specifically, the second training set includes face images and labels.

[0091] Preferably, the marking method is as follows: for a single-frame image of a living user captured by a camera, it is marked as a living object; for any single-frame image in an action set of a user's prosthetic object (such as a photo, video, or three-dimensional mold, etc.) captured by the camera, it is marked as a prosthetic object; for each living object sample and prosthetic image sample in the first training set, an adversarial sample is generated and marked as a prosthetic object.

[0092] Generating adversarial examples is very time-consuming. Therefore, using too high a ratio of adversarial examples will slow convergence during training. However, significantly reducing the ratio of adversarial examples will prevent the model from fully learning robust features. Preferably, in this embodiment, the ratio of images of living subjects, images of prosthetic subjects, and adversarial examples is 1:1:2.

[0093] Step S500: constructing a defense training loss function for face liveness detection, and using the second training set to iteratively optimize the deep neural network to obtain a robust face liveness detection classification model.

[0094] Specifically, we construct a defense training loss function for face liveness detection, which is formalized as follows:

[0095] ,

[0096] in, represents the loss function for the classification problem, represents the adversarial metric learning loss function, represents the regularization term, , and are the weight coefficients of each loss term.

[0097] Based on the observation and comparison of a large number of experiments in this invention, this embodiment is preferred =2, =1, =0.002. In practical applications, the hyperparameters can be adjusted according to specific circumstances using the grid search method.

[0098] Specifically, the classification loss function The formal representation is as follows:

[0099]

[0100] in, represents the binary classification loss function, a face image representing a living subject, x represents the secondary image of the prosthetic object, represents adversarial samples, Indicates that the category label is living, Indicates that the category label is a pseudo-body. Thus, a three-category problem is simplified to a two-category problem to reduce the complexity of the problem.

[0101] Preferably, this embodiment uses cross entropy as the binary classification loss function.

[0102] Specifically, adversarial metric learning loss function The formal representation is as follows:

[0103]

[0104] Where T represents the triplet loss function, g(·) represents the representation of the input image by the deep neural network, Represents the deep neural network representation of input image x1 and input image x2 and The distance between them, a is the preset boundary value, and max{·} indicates the maximum operation.

[0105] It is understandable that for conventional face liveness detection tasks, only conventional live object samples and conventional prosthetic image samples are involved. It is a standard two-classification problem, and the use of pair-wise contrast learning can achieve ideal results. However, the main problem to be solved by the present invention is to detect adversarial samples of face images. Therefore, in addition to the conventional pixel-based classification task training, the present application proposes training in the latent feature space. Specifically, by minimizing the triplet loss function, the distance between the adversarial samples and the conventional prosthetic image samples is shortened (the two are of the same type: prosthetic objects), and the distance between them and the live object samples is increased (prosthetic objects and live objects are of different types). This is a better design.

[0106] Preferably, this embodiment uses the output of the last convolutional layer of the deep neural network ResNet18 as the representation , to obtain higher-level and richer semantic information.

[0107] Preferably, this embodiment uses angular distance as the distance function D in the triplet loss function, and the preset boundary value a=0.1.

[0108] Preferably, the regularization term used in this embodiment is The formal representation is as follows:

[0109]

[0110] in, represents regularization with a norm of p. Preferably, this embodiment adopts p=2 to avoid excessive amplitude of features that may lead to unstable training process.

[0111] Preferably, the iterative optimization training settings are as follows: maximum number of iterations is 10,000; batch size is 64; decay coefficient is 0.0002, and momentum term is 0.9. If computing power is sufficient, the training convergence can be accelerated by increasing the batch size.

[0112] Specifically, the robust face liveness detection and classification model can not only distinguish between the image of a live object and the secondary imaging of a prosthetic object, but also correctly classify adversarial samples as prosthetic objects, that is,

[0113] ,and ,and ,

[0114] Thus, a three-class classification problem is simplified to a two-class classification problem on each category.

[0115] Example 2:

[0116] Corresponding to the aforementioned embodiment of the method for constructing a face liveness detection classification model, the present application also provides an embodiment of an apparatus for constructing a face liveness detection classification model.

[0117] Figure 3 This is a block diagram of a device for constructing a face liveness detection classification model according to an exemplary embodiment. Figure 3 , the device comprises:

[0118] A first acquisition module 11 is used to acquire training samples;

[0119] A first building block 12 is used to build a deep neural network for face liveness detection;

[0120] A second construction module 13 is used to preprocess the training samples to construct a first training set;

[0121] A third construction module 14 is configured to generate a corresponding adversarial sample for each sample in the first training set and add the sample to the first training set to form a second training set;

[0122] The model construction module 15 is used to construct a defense training loss function for face liveness detection, and use the second training set to iteratively optimize the deep neural network to obtain a robust face liveness detection classification model.

[0123] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0124] Example 3:

[0125] refer to Figure 4 , the embodiment of the present application provides a method for face liveness detection and classification, comprising:

[0126] Step S21, obtaining the original test data and performing preprocessing;

[0127] Specifically, obtaining the data to be tested includes:

[0128] Determine the form of the data to be tested;

[0129] If the acquired test data is image data, then directly detect whether the image contains a human face. If so, preprocess the image; if not, do not process it. If the acquired test data is video data, extract a single-frame image from the video sequence as the test image, detect whether the test image contains a human face, and if so, preprocess the image. If not, continue to extract single-frame images from the video sequence to be tested, detect whether the single-frame image contains a human face, and repeat this process until a face is detected in the single-frame image, and preprocess the single-frame image.

[0130] Specifically, the preprocessing includes: facial key point detection, image enhancement and image normalization.

[0131] Step S22: input the data to be tested into the above-mentioned face liveness detection classification model and output the detection result.

[0132] The data to be tested is input into the robust face liveness detection classification model to detect whether the current face object is a live object. If not, the current data is rejected from entering the face recognition system.

[0133] It should be understood that the present invention is not limited by the specific face detection, face key point detection, image enhancement, image normalization method, conventional image classification task training method and adversarial sample generation method adopted. Whether it is the existing face detection, face key point detection, image enhancement, image normalization method, conventional image classification task training method and adversarial sample generation method or the face detection, face key point detection, image enhancement, image normalization method and adversarial sample generation method developed in the future, they can all be applied to the face liveness detection method according to the embodiment of the present invention, and should also be included in the protection scope of the present invention.

[0134] Example 4:

[0135] Corresponding to the aforementioned embodiment of the human face liveness detection and classification method, the present application also provides an embodiment of a human face liveness detection and classification device.

[0136] FIG5 is a block diagram of a face liveness detection and classification device according to an exemplary embodiment. Figure 5 , the device comprises:

[0137] The second acquisition module 21 is used to acquire the original test data and perform preprocessing;

[0138] The detection module 22 is used to input the data to be tested into the above-mentioned face liveness detection classification model and output the detection results.

[0139] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0140] Example 5:

[0141] Please refer to Figure 6 , Figure 6 This is a block diagram of a facial liveness detection and classification system provided in Example 3 of the present invention. The system includes the facial liveness detection and classification device 400 and the identity recognition system 500. The adversarial sample defense device for facial liveness detection and the identity recognition system are interconnected, and raw data is input into the identity recognition system after passing through the adversarial sample defense device for facial liveness detection.

[0142] In summary, the adversarial sample defense method, device and system for face liveness detection proposed in each embodiment of the present invention combine metric learning under conventional classification training with adversarial classification training. In this way, it can not only solve the problem of low classification accuracy caused by the very small class distance between images of living objects and prosthetic objects under conventional two-classification method, but also effectively overcome the interference caused by adversarial samples to the face liveness detection classification model. By preventing adversarial samples carefully designed by illegal users from being input into the face recognition system, it ensures that the identity recognition system that includes the face liveness detection function can process the input data normally without being interfered by adversarial samples.

[0143] Example 6:

[0144] Accordingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method for constructing a face liveness detection classification model or the face liveness detection classification method. Figure 7 As shown in FIG, a hardware structure diagram of a device for constructing a face liveness detection and classification model provided by an embodiment of the present invention or a device with data processing capability where the face liveness detection and classification device is located, except for Figure 7 In addition to the processor and memory shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware according to the actual functions of the device with data processing capabilities, which will not be described in detail.

[0145] Accordingly, the present application also provides a computer-readable storage medium having computer instructions stored thereon, characterized in that, when executed by a processor, the instructions implement the above-described method for constructing a face liveness detection and classification model or the face liveness detection and classification method. The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device of a wind turbine, such as a plug-in hard disk, a Smart Media Card (SMC), an SD card, a flash memory card, etc. equipped on the device. Furthermore, the computer-readable storage medium may include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0146] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered merely as exemplary, and the true scope and spirit of the present application are indicated by the claims.

[0147] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0148] The method for constructing a face liveness detection classification model and the face liveness detection classification method involved in the specific implementation of this application, when the above embodiments of this application are applied to specific products or technologies, need to obtain user permission or consent, and the collection, use and processing of relevant information need to comply with relevant laws, regulations and standards of relevant countries and regions.

Claims

1. A method for constructing a face liveness detection classification model, characterized in that: include: Get training samples; Build a deep neural network for face liveness detection; Preprocessing the training samples to construct a first training set; For each sample in the first training set, generate a corresponding adversarial sample and add it to the first training set to form a second training set; Constructing a defense training loss function for face liveness detection, and iteratively optimizing the deep neural network using the second training set to obtain a robust face liveness detection classification model; The defensive training loss function L for face liveness detection is formally expressed as follows: ， in, represents the loss function for the classification problem, represents the adversarial metric learning loss function, represents the regularization term, , and are the weight coefficients of each loss term; The loss function for the classification problem is The formal representation is as follows: ， in represents a deep neural network, represents the binary classification loss function, a face image representing a living subject, represents a secondary image of a prosthetic object, Represents a face image Superimpose adversarial disturbances Post-synthesized adversarial examples, Indicates that the category label is living, Indicates that the category label is prosthesis; The adversarial metric learning loss function The formal representation is as follows: ， Where T represents the triplet loss function, g(·) represents the representation of the input image by the deep neural network, Represents adversarial examples and secondary imaging of prosthetic objects Deep neural network representation and The distance between them, a is the preset boundary value; The regularization term The formal representation is as follows: ， in, represents the regularization with norm p; By minimizing the triplet loss function, we narrow the distance between similar adversarial samples and fake image samples, and increase the distance between them and non-similar living object samples; The robust face liveness detection and classification model can not only distinguish between the image of a live object and the secondary imaging of a prosthetic object, but also correctly classify adversarial samples as prosthetic objects, i.e., ,and ,and .

2. The method according to claim 1, wherein The adversarial example is formally represented as follows: ， in, represents the vector representation of the adversarial perturbation, , Indicates the imperceptibility of the disturbance, Represents the face image in the regular sample training set, which is the image of a living object or the secondary imaging of a prosthetic object. Representing a deep neural network Adversarial examples A face image classified as a living object, that is, the current detected object is judged to be a living object.

3. A device for constructing a face liveness detection classification model, comprising: A first acquisition module is used to acquire training samples; The first building block is used to build a deep neural network for face liveness detection; A second construction module is used to preprocess the training samples to construct a first training set; A third construction module is configured to generate a corresponding adversarial sample for each sample in the first training set and add the sample to the first training set to form a second training set; A model construction module is used to construct a defensive training loss function for face liveness detection, and iteratively optimize the deep neural network using the second training set to obtain a robust face liveness detection classification model; The defensive training loss function L for face liveness detection is formally expressed as follows: ， in, represents the loss function for the classification problem, represents the adversarial metric learning loss function, represents the regularization term, , and are the weight coefficients of each loss term; The loss function for the classification problem is The formal representation is as follows: ， in represents a deep neural network, represents the binary classification loss function, a face image representing a living subject, represents a secondary image of a prosthetic object, Represents a face image Superimpose adversarial disturbances Post-synthesized adversarial examples, Indicates that the category label is living, Indicates that the category label is prosthesis; The adversarial metric learning loss function The formal representation is as follows: ， Where T represents the triplet loss function, g(·) represents the representation of the input image by the deep neural network, Represents adversarial examples and secondary imaging of prosthetic objects Deep neural network representation and The distance between them, a is the preset boundary value; The regularization term The formal representation is as follows: ， in, represents the regularization with norm p; By minimizing the triplet loss function, we narrow the distance between similar adversarial samples and fake image samples, and increase the distance between them and non-similar living object samples; The robust face liveness detection and classification model can not only distinguish between the image of a live object and the secondary imaging of a prosthetic object, but also correctly classify adversarial samples as prosthetic objects, i.e., ,and ,and .

4. A method for detecting and classifying living faces, characterized in that: include: Obtain the original test data and perform preprocessing; The data to be tested is input into the face liveness detection classification model described in claim 1, and the detection result is output.

5. The method according to claim 4, wherein Obtain the data to be tested and perform preprocessing, including: Determine the form of the data to be tested; If the acquired test data is image data, then directly detect whether the image contains a human face. If so, preprocess the image; if not, do not process it. If the acquired test data is video data, extract a single-frame image from the video sequence as the test image, detect whether the test image contains a human face, and if so, preprocess the image. If not, continue to extract single-frame images from the video sequence to be tested, detect whether the single-frame image contains a human face, and repeat this process until a face is detected in the single-frame image, and preprocess the single-frame image.

6. A face liveness detection and classification device, characterized in that: include: The second acquisition module is used to obtain the original test data and perform preprocessing; The detection module is used to input the data to be tested into the face liveness detection classification model described in claim 1 and output the detection results.

7. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-2, 4-5.

8. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1-2, 4-5 are implemented.

Images