Training methods for liveness detection models, liveness detection methods and devices
By combining the pre-training of the feature extraction module and the image restoration module, the problem of poor generalization ability of the liveness detection algorithm in long-distance face detection is solved, and the clear processing of blurry images and the accuracy improvement are achieved, which is suitable for liveness detection in remote monitoring scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DUXIAOMAN TECH (BEIJING) CO LTD
- Filing Date
- 2023-08-28
- Publication Date
- 2026-07-17
AI Technical Summary
Existing liveness detection algorithms have poor generalization ability and low accuracy when detecting faces at long distances, and are difficult to effectively cope with the effects of motion blur and occlusion.
By pre-training the feature extraction module, restoring blurred images using the image restoration module, classifying them using the classification module, and training the liveness detection model using a multi-loss fusion module, the model's generalization ability and accuracy are improved.
The liveness detection model has been enhanced in its ability to handle blurred images, improving its generalization ability and accuracy, making it suitable for liveness detection in remote monitoring scenarios.
Smart Images
Figure CN117173797B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of liveness detection technology, and particularly relates to a training method for a liveness detection model, a liveness detection method, and an apparatus. Background Technology
[0002] Existing liveness detection technologies, whether based on color and texture feature learning, facial depth structure fitting, or remote photoplethysmography (rPPG), all require high-quality image details to ensure high performance. However, when detecting faces at long distances, these algorithms suffer from poor generalization ability and low accuracy due to the small resolution of faces and the influence of motion blur, occlusion, and other noise factors. Summary of the Invention
[0003] This application provides an implementation scheme that differs from related technologies, in order to solve the technical problems of poor generalization ability and low accuracy of liveness detection algorithms in related technologies.
[0004] Firstly, this application provides a training method for a liveness detection model. The liveness detection model includes an image restoration module, a feature extraction module, and a classification module. The classification module consists of fully connected layers and is used to determine whether the object corresponding to the input information is a real person or a non-real person. The method includes:
[0005] The parameters of the feature extraction module are pre-trained to obtain the initial parameters of the feature extraction module;
[0006] Obtain a first image from the image sample set, and a second image from the first image after blurring.
[0007] The second image is restored using the image restoration module to obtain the restored image;
[0008] The feature extraction module extracts features from the restored image to obtain the corresponding facial feature information;
[0009] The classification module categorizes the input facial feature information to obtain the predicted classification result;
[0010] The liveness detection model is trained based on the first image, the restored image, and the predicted classification results to obtain a trained liveness detection model.
[0011] Secondly, this application provides a method for detecting liveness, the method comprising:
[0012] The image restoration module in the trained liveness detection model is used to restore the image to be detected, and the restored image is obtained.
[0013] The feature extraction module in the trained liveness detection model is used to extract features from the restored image to obtain the corresponding facial feature information.
[0014] The classification module in the trained liveness detection model classifies the input facial feature information to obtain the detection result, which is used to indicate whether the object in the image to be detected is a real person or a non-real person.
[0015] The initial parameters of the feature extraction module are obtained through pre-training.
[0016] Thirdly, this application provides a training device for a liveness detection model. The liveness detection model includes an image restoration module, a feature extraction module, and a classification module. The classification module consists of fully connected layers and is used to determine whether the object corresponding to the input information is a real person or a non-real person. The device includes:
[0017] The pre-training unit is used to pre-train the parameters of the feature extraction module to obtain the initial parameters of the feature extraction module.
[0018] The acquisition unit is used to acquire a first image from an image sample set, and a second image after blurring the first image;
[0019] The restoration unit is used to perform restoration processing on the second image through the image restoration module to obtain the restored image;
[0020] The feature extraction unit is used to extract features from the restored image through the feature extraction module to obtain the corresponding facial feature information;
[0021] The classification unit is used to classify the input facial feature information through the classification module to obtain the predicted classification result;
[0022] The training unit is used to train the liveness detection model based on the first image, the restored image, and the predicted classification results to obtain a trained liveness detection model.
[0023] Fourthly, this application provides an electronic device, comprising:
[0024] Processor; and
[0025] Memory for storing the executable instructions of the processor;
[0026] The processor is configured to execute the first aspect, the second aspect, any possible implementation of the first aspect, or any method in any possible implementation of the second aspect by executing the executable instructions.
[0027] Fifthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the first aspect, the second aspect, and any possible implementation of the first aspect, or any method in any possible implementation of the second aspect.
[0028] This application provides a scheme for pre-training the parameters of the feature extraction module to obtain initial parameters for the feature extraction module; acquiring a first image and a second image (after blurring) from an image sample set; restoring the second image using the image restoration module to obtain a restored image; extracting features from the restored image using the feature extraction module to obtain corresponding facial feature information; classifying the input facial feature information using the classification module to obtain a predicted classification result; and training the liveness detection model based on the first image, the restored image, and the predicted classification result to obtain a trained liveness detection model. This scheme utilizes the pre-training of the feature extraction module's parameters, using the pre-trained parameters as initial values for the feature extraction module to fine-tune the liveness detection model, improving its training effect. The image restoration module restores blurred images, enhancing low-quality facial data and helping the liveness detection model to some extent clarify blurred images, thereby improving the generalization ability and accuracy of the liveness detection model. Attached Figure Description
[0029] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:
[0030] Figure 1 A schematic flowchart illustrating a training method for a liveness detection model provided in an embodiment of this application;
[0031] Figure 2 A schematic diagram illustrating the principle of a training method for a pre-trained model provided in an embodiment of this application;
[0032] Figure 3 This is a schematic diagram of the structure of an image restoration module provided in one embodiment of this application;
[0033] Figure 4 A schematic diagram illustrating the relationship between a liveness detection model and a pre-trained model provided in an embodiment of this application;
[0034] Figure 5A schematic diagram of the structure of a training device for a liveness detection model provided in an embodiment of this application;
[0035] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0036] The embodiments of this application are described in detail below, with examples of the embodiments shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. It is understood that in the specific embodiments of this application, data related to image sample sets, unlabeled image sample sets, etc., are involved. When the above embodiments of this application are applied to specific products or technologies, user permission, consent, or authorization is required, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.
[0037] The terms "first" and "second," etc., used in the specification and drawings of embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the solution can be implemented in a different order than that illustrated or described in this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.
[0038] First, some terms used in the embodiments of this application will be explained below to facilitate understanding by those skilled in the art.
[0039] MAE: masked autoencoders.
[0040] GAN: Generative Adversarial Networks.
[0041] CAE (Context Autoencoder) is a masked image modeling method for scalable self-supervised representation learning pre-training. The core idea involves predicting from visible blocks to masked blocks in the latent representation space; the encoder is used solely for representation learning, and representation learning is performed only by the encoder.
[0042] Gaussian blur, also known as Gaussian smoothing, is a commonly used technique in image processing, primarily used to reduce image noise and detail.
[0043] Motion blur: Motion blur is a noticeable blurring or dragging effect caused by fast-moving objects in a static scene or a series of images, such as in a movie or animation.
[0044] Face liveness detection refers to detecting whether a given face image contains a real face, rather than a fake or reproduced image. In recent years, with the development of deep learning technology and the increasing use cases for face detection, although close-range face liveness detection technology (such as selfies) has become increasingly mature, its accuracy remains relatively low in surveillance scenarios due to factors such as face resolution, angle, and occlusion. With technological advancements, increased public safety risks, and higher user experience requirements, remote monitoring scenarios have emerged. In these scenarios, user cooperation is not required (e.g., looking directly at the camera, removing a mask), and the differences in face angles and distances further exacerbate the problem. Consequently, the generalization ability of liveness detection models in related technologies is poor, resulting in low accuracy and difficulty in achieving ideal results.
[0045] To address this technical problem, this application provides a training method for a liveness detection model, a liveness detection method, and an apparatus, which solve the technical problems of poor generalization ability and low accuracy of liveness detection algorithms in related technologies.
[0046] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.
[0047] Figure 1 This is a flowchart illustrating a training method for a liveness detection model provided as an exemplary embodiment of this application. The liveness detection model includes an image restoration module, a feature extraction module, and a classification module. The classification module consists of fully connected layers and is used to determine whether the object corresponding to the input information is a real person or a non-real person. This method can be applied to training devices and includes at least the following steps S11-S16:
[0048] S11. Pre-train the parameters of the feature extraction module to obtain the initial parameters of the feature extraction module;
[0049] The feature extraction module is used to extract facial features from the face image corresponding to the input information.
[0050] In some embodiments, the input information is the face image itself.
[0051] In some embodiments, in S11, the parameters of the feature extraction module are pre-trained to obtain the initial parameters of the feature extraction module, including the following S111-S112:
[0052] S111. Train the initial pre-trained model using the unlabeled image sample set to obtain the target pre-trained model;
[0053] In some embodiments, the unlabeled image sample set can be any open-source face dataset, and includes a large number of samples.
[0054] In some specific implementations, an unlabeled face sample set of approximately 5 million can be constructed using open-source datasets such as CASIA-SURF, CelebA, and self-collected face datasets as an unlabeled image sample set.
[0055] Figure 2 A schematic diagram illustrating the principle of a training method for a pre-trained model provided as an exemplary embodiment of this application is shown below. Figure 2 The pre-trained model includes a variational encoder, a decoder, an image encoder, and a softmax classifier.
[0056] In some embodiments, in the pre-trained model, for the original images of each two-dimensional image in the unlabeled image sample set, the original images of each two-dimensional image are first divided into patch grids to obtain a preset number of patch grids; further, the preset number of patch grids are processed in the following two parts, namely the first part and the second part.
[0057] Part 1: A preset number of patch grids are encoded by a tokenizer variational encoder to obtain a preset number of visual token image markers. Each visual token corresponds to a patch grid. The visual token is another way of representing a two-dimensional image and contains image information. Further, the preset number of visual tokens are encoded by a decoder to obtain the reconstructed image of the two-dimensional image.
[0058] Part Two: For a preset number of patch grids, randomly select a few patch grids and perform masking operations to obtain a third image including the masked image patches. See [link to documentation]. Figure 2The black image patches in the third image are the occluded image patches. Further, each patch grid in the third image is mapped to a one-dimensional vector through linear mapping, making each patch grid a vector. A predetermined number of patch grids are then flattened to obtain a two-dimensional matrix, which is used as input to the image encoder. The two-dimensional matrix is then input into the image encoder for image encoding, generating an image feature vector, which is then passed through a softmax classifier to output a multi-dimensional vector.
[0059] Furthermore, based on the multidimensional vector output by the softmax classifier, multiple predicted visual tokens corresponding to the occluded patch grids in the two-dimensional image are obtained. These predicted visual tokens are then compared with the image labels of the visual tokens corresponding to the occluded patch grids obtained in the first part, and the softmax loss is calculated. The mean squared error (MSE) loss is calculated based on the reconstructed image and the original image. Based on the softmax loss, MSE loss, and gradient method, the variational encoder, decoder, image encoder, and softmax classifier in the initial pre-trained model are trained to obtain the target pre-trained model. The training process of the target pre-trained model will not be elaborated here.
[0060] In a specific embodiment provided in this application, the image encoder is an encoder based on the ViT network, and the softmax classifier is a Mask Image Modeling Head composed of fully connected layers. The training method of the pre-trained model is described below with a specific embodiment.
[0061] First, the original 224*224 image is patched to obtain 14*14 patch grids, where each patch grid is 16*16 in size.
[0062] Furthermore, for the 14*14 patch grid, the following two operations are performed.
[0063] Part 1: Decode the 14*14 patch grids using a tokenizer to obtain 14*14 visual tokens. Each visual token is a value between [1-8192]. Then, encode the 14*14 visual tokens using a decoder to obtain the reconstructed image.
[0064] Part Two: Randomly occlude 14*14 patch grids to obtain a third image containing multiple occluded image patches. Map the 14*14 patch grids to a one-dimensional vector using linear mapping. Each patch grid with a data shape of [16,16,3] is mapped to a vector of length 768. Flattening the vector corresponding to each patch grid results in [14,14,768]->[196,768], which becomes a two-dimensional matrix, serving as the input to the ViT network image encoder. Further, this two-dimensional matrix is input to the image encoder to obtain image feature vectors, which are then mapped to H vectors {h1,…hP} via a softmax classifier (fully connected layer). Each patch corresponds to an h vector, and the index of the maximum value in each h vector is the predicted visual token, i.e., the predicted visual token of the occluded image patch.
[0065] Furthermore, based on the comparison between the predicted visual token and the visual token image labels corresponding to the multiple occluded patch grids obtained in the first part, the softmax loss is obtained. The MSE loss is calculated based on the reconstructed image and the original image. Based on the softmax loss and the MSE loss, the gradient method is used to complete the training of the variational encoder, decoder, image encoder and softmax classifier in the initial pre-trained model, and the target pre-trained model is obtained.
[0066] In some embodiments, in addition to using the methods described above to train the initial pre-trained model, methods such as MAE and CAE can also be used to train the initial pre-trained model.
[0067] S112. Use the parameters of the encoding unit in the target pre-trained model as the initial parameters of the feature extraction module, wherein the network structure of the encoding unit is the same as that of the feature extraction module.
[0068] Specifically, the encoding unit of the target pre-trained model is the image encoder mentioned above.
[0069] By training the initial pre-trained model on a large scale, the target pre-trained model learns many useful features. Using the parameters of the target pre-trained model as the initial parameters of the feature extraction module to fine-tune the liveness detection model can achieve good results.
[0070] S12. Obtain a first image from the image sample set, and a second image after blurring the first image;
[0071] In some embodiments, the first image is blurred using Gaussian blurring.
[0072] In other embodiments, the first image is blurred by using a preset algorithm to process the first image, so that the first image has a motion blur effect.
[0073] The first image has higher resolution than the second image.
[0074] S13. The second image is restored using the image restoration module to obtain the restored image;
[0075] In some embodiments, the image restoration module employs a NAFNet (Nonlinear Activation Free Network) to enhance and restore low-quality face images.
[0076] In other embodiments, the network structure of the image restoration module can also be constructed using the GAN method.
[0077] Figure 3 This is a schematic diagram of the structure of an image restoration module provided in an exemplary embodiment of this application. The image restoration module includes computational structures such as LayerNorm, conv, deconv, Simple, and Channel Attention. Specifically, Channel Attention includes three computational structures: conv, GELU, and conv. The specific implementation principles of computational structures such as LayerNorm, conv, deconv, Simple, and Channel Attention can be found in the relevant content of NAFNet, and will not be elaborated here.
[0078] In surveillance scenarios, due to the distance between people and cameras and the movement of people, some facial samples may be blurry. Adding an image restoration module can enhance poor-quality facial images to a certain extent, deblurring blurry images to obtain relatively clear facial images, which is beneficial for subsequent classification modules to detect and recognize.
[0079] S14. The feature extraction module extracts features from the restored image to obtain the corresponding facial feature information;
[0080] In some embodiments, the feature extraction module is used to extract features from the face image in the input information. It can automatically extract multi-dimensional and deep-level face image features and output a 4096-dimensional feature vector of the restored image.
[0081] In some embodiments, in addition to using the encoding unit in the target pre-trained model as a feature extraction module, other convolutional neural networks for extracting feature information from face images can also be used, such as ResNet, SENet, etc.
[0082] S15. The input facial feature information is classified through the classification module to obtain the predicted classification result;
[0083] In some embodiments, the classification module is a fully connected layer of 4096*2.
[0084] Specifically, the predicted classification result may include whether the detected object corresponding to the restored image corresponding to the facial feature information is a real person or a non-real person, that is, whether the detected object corresponding to the first image is a real person or a non-real person.
[0085] S16. Based on the first image, the restored image, and the predicted classification results, the liveness detection model is trained to obtain a trained liveness detection model.
[0086] In some embodiments, in S16, the liveness detection model is trained based on the first image, the restored image, and the predicted classification result, including the following S161-S163:
[0087] S161. Determine the restoration loss based on the first image and the restored image;
[0088] In some embodiments, in S161, determining the restoration loss based on the first image and the restored image includes: calculating the restoration loss between the first image and the restored image using the mean square error method.
[0089] Specifically, for image sample sets, the formula can be used. Calculate the restoration loss L of the image sample set MSE Where n represents the number of image samples in the image sample set, and y is the pixel value of the restored image. These are the pixel values of the first image.
[0090] S162. Determine the classification loss based on the predicted classification results;
[0091] In some embodiments, the predicted classification result may include whether the detected object corresponding to the restored image corresponding to the facial feature information is a real person or a non-real person, and the classification loss can be determined based on the predicted classification result and the true label of the first image.
[0092] Specifically, formula L can be used. c = -(p*log(q)+(1-p)*log(1-q)) calculates the classification loss L between the predicted classification result and the true label of the first image. c , where p is the true label and q is the predicted classification result.
[0093] S163. Train the liveness detection model based on restoration loss and classification loss.
[0094] In some embodiments, the liveness detection model further includes a multi-loss fusion module. In S163, the liveness detection model is trained based on the recovery loss and the classification loss, including S1631-S1632:
[0095] S1631. The fusion loss is obtained by weighted summation of the restoration loss and classification loss through the multi-loss fusion module.
[0096] In some embodiments, the weights of the recovery loss and the classification loss are 0.1 and 0.9, respectively.
[0097] S1632. When the fusion loss is less than the preset value, the most recently determined liveness detection model is used as the trained liveness detection model. When the fusion loss is not less than the preset value, the parameters of the liveness detection model are adjusted based on the restoration loss and classification loss, and the process returns to the step of obtaining the first image from the image sample set to continue training the liveness detection model.
[0098] The multi-loss fusion module integrates the cross-entropy classification loss L... c Recovery loss L MSE This is equivalent to two tasks mutually reinforcing each other's learning of the liveness detection model, which can effectively improve the accuracy of the liveness detection model in liveness detection.
[0099] Figure 4 This is a schematic diagram illustrating the relationship between a liveness detection model and a pre-trained model provided in an exemplary embodiment of this application. The pre-trained model includes an encoder and a decoder. The input is a third image in which the original face image is divided into a preset number of patch grids and the image patches are randomly occluded. The occluded image patches can be predicted through the pre-trained model to obtain a reconstructed image.
[0100] The liveness detection model includes: an image restoration module, a feature extraction module, a classification module, and a multi-loss fusion module, among which:
[0101] The input to the image restoration module is a second image obtained by blurring the first image. The second image can be restored to obtain a restored image. Through the image restoration module, images with low clarity and poor quality can be restored, thereby enhancing the image quality of the input information.
[0102] The input to the feature extraction module is the restored image, and the initial parameters of the feature extraction module are the parameters of the encoding unit (i.e., the image encoder) in the pre-trained target model. The feature extraction module is used to extract the feature information of the face image corresponding to the input information.
[0103] The classification module consists of fully connected layers and is used to determine whether the object corresponding to the face image of the input information is a real person or a non-real person.
[0104] The multi-loss fusion module is used to calculate the fusion loss based on the restoration loss between the first image and the restored image in the image restoration module and the classification loss between the predicted classification result and the true label in the classification module. Based on the fusion loss and the gradient method, the parameters of the image restoration module, the feature extraction module and the classification module are updated, i.e., the liveness detection model is trained.
[0105] In one embodiment, this application also provides a liveness detection method, the method comprising:
[0106] The image restoration module in the trained liveness detection model is used to restore the image to be detected, and the restored image is obtained.
[0107] The feature extraction module in the trained liveness detection model is used to extract features from the restored image to obtain the corresponding facial feature information.
[0108] The classification module in the trained liveness detection model classifies the input facial feature information to obtain the detection result, which is used to indicate whether the object in the image to be detected is a real person or a non-real person.
[0109] The initial parameters of the feature extraction module are obtained through pre-training.
[0110] In some embodiments, the application scenarios of this application can be applied to tasks such as clocking in and retrieving items.
[0111] For specific details of the embodiments described above, please refer to the foregoing content. For details of the pre-training method, please refer to the foregoing content. They will not be repeated here.
[0112] This application provides a scheme for pre-training the parameters of a feature extraction module to obtain initial parameters; acquiring a first image and a second image (after blurring) from an image sample set; restoring the second image using an image restoration module to obtain a restored image; extracting features from the restored image using the feature extraction module to obtain corresponding facial feature information; classifying the input facial feature information using a classification module to obtain a predicted classification result; and training a liveness detection model based on the first image, the restored image, and the predicted classification result to obtain a trained liveness detection model. This scheme utilizes the pre-trained parameters of the feature extraction module, using these pre-trained parameters as initial values for fine-tuning the model, thus improving the training effect. The image restoration module restores blurred images, enhancing low-quality facial data and enabling the liveness detection model to clear blurred images to a certain extent, thereby improving the generalization ability and accuracy of the liveness detection model.
[0113] Figure 5 This is a schematic diagram of the structure of a training device for a liveness detection model provided as an exemplary embodiment of this application. The liveness detection model includes an image restoration module, a feature extraction module, and a classification module. The classification module is composed of fully connected layers and is used to determine whether the object corresponding to the input information is a real person or a non-real person.
[0114] The device includes:
[0115] The pre-training unit 51 is used to pre-train the parameters of the feature extraction module to obtain the initial parameters of the feature extraction module.
[0116] The acquisition unit 52 is used to acquire a first image from the image sample set and a second image of the first image after blurring.
[0117] The restoration unit 53 is used to perform restoration processing on the second image through the image restoration module to obtain the restored image;
[0118] The feature extraction unit 54 is used to extract features from the restored image through the feature extraction module to obtain the corresponding facial feature information;
[0119] Classification unit 55 is used to classify the input facial feature information through the classification module to obtain the predicted classification result;
[0120] Training unit 56 is used to train the liveness detection model based on the first image, the restored image, and the predicted classification results to obtain a trained liveness detection model.
[0121] In some embodiments, when the device is used to pre-train the parameters of the feature extraction module to obtain the initial parameters of the feature extraction module, it is specifically used for:
[0122] The initial pre-trained model is trained using an unlabeled image sample set to obtain the target pre-trained model;
[0123] The parameters of the encoding units in the target pre-trained model are used as the initial parameters of the feature extraction module, where the encoding units have the same network structure as the feature extraction module.
[0124] In some embodiments, when the apparatus is used to train a liveness detection model based on a first image, a restored image, and a predicted classification result, it is specifically used for:
[0125] The restoration loss is determined based on the first image and the restored image;
[0126] Determine the classification loss based on the predicted classification results;
[0127] The liveness detection model is trained based on restoration loss and classification loss.
[0128] In some embodiments, the liveness detection model further includes a multi-loss fusion module, which, when used to train the liveness detection model based on recovery loss and classification loss, is specifically used for:
[0129] The fusion loss is obtained by weighted summation of the restoration loss and classification loss through the multi-loss fusion module.
[0130] When the fusion loss is less than the preset value, the most recently determined liveness detection model is used as the trained liveness detection model. When the fusion loss is not less than the preset value, the parameters of the liveness detection model are adjusted based on the restoration loss and classification loss, and the process returns to the step of obtaining the first image from the image sample set to continue training the liveness detection model.
[0131] In some embodiments, when the apparatus is used to determine the restoration loss based on the first image and the restored image, it is specifically used to:
[0132] The mean square error method is used to calculate the restoration loss between the first image and the restored image.
[0133] In some embodiments, the device performs Gaussian blurring on the first image.
[0134] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, they will not be repeated here. Specifically, the device can execute the above method embodiments, and the foregoing and other operations and / or functions of each module in the device correspond to the corresponding processes in the various methods in the above method embodiments, which will not be repeated here for the sake of brevity.
[0135] The apparatus of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.
[0136] Figure 6 This is a schematic block diagram of an electronic device provided in an embodiment of this application. The electronic device may include:
[0137] The system includes a memory 601 and a processor 602. The memory 601 stores computer programs and transfers the program code to the processor 602. In other words, the processor 602 can retrieve and run the computer programs from the memory 601 to implement the methods described in the embodiments of this application.
[0138] For example, the processor 602 can be used to execute the above-described method embodiments according to instructions in the computer program.
[0139] In some embodiments of this application, the processor 602 may include, but is not limited to:
[0140] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0141] In some embodiments of this application, the memory 601 includes, but is not limited to:
[0142] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).
[0143] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 601 and executed by the processor 602 to perform the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.
[0144] like Figure 6 As shown, the electronic device may further include:
[0145] Transceiver 603, which can be connected to processor 602 or memory 601.
[0146] The processor 602 can control the transceiver 603 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 603 may include a transmitter and a receiver. The transceiver 603 may further include antennas, and the number of antennas may be one or more.
[0147] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.
[0148] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.
[0149] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0150] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0151] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.
[0152] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
[0153] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A training method for a liveness detection model, characterized in that, The liveness detection model includes an image restoration module, a feature extraction module, and a classification module. The classification module, composed of fully connected layers, is used to determine whether the input information corresponds to a real person or a non-real person. The method includes: The parameters of the feature extraction module are pre-trained to obtain the initial parameters of the feature extraction module; Obtain a first image from the image sample set, and a second image that has been blurred from the first image; The image restoration module performs restoration processing on the second image to obtain the restored image; The feature extraction module extracts features from the restored image to obtain corresponding facial feature information. The classification module classifies the input facial feature information to obtain a predicted classification result. The liveness detection model is trained based on the first image, the restored image, and the predicted classification result to obtain a trained liveness detection model. The parameters of the feature extraction module are pre-trained to obtain the initial parameters of the feature extraction module, including: The initial pre-trained model is trained using an unlabeled image sample set to obtain the target pre-trained model; The parameters of the encoding unit in the target pre-trained model are used as the initial parameters of the feature extraction module, wherein the encoding unit has the same network structure as the feature extraction module; The step of training the initial pre-trained model using an unlabeled image sample set to obtain the target pre-trained model includes: The original images in the unlabeled image sample set are divided into multiple image blocks; The multiple image blocks are processed through the first processing branch: each image block is encoded using a variational encoder to obtain a visual marker corresponding to each image block; then the visual markers are decoded using a decoder to obtain a reconstructed image of the original image; The second processing branch processes the multiple image patches: a portion of the multiple image patches is masked to obtain a third image containing the masked image patches; each image patch in the third image is mapped into a vector and flattened into a two-dimensional matrix, which is then input into an image encoder to generate an image feature vector; the image feature vector is then output through a softmax classifier to predict the visual label corresponding to the masked image patch. Based on the softmax loss between the visual markers predicted by the second processing branch and the visual markers corresponding to the occluded image blocks in the first processing branch, and the mean square error loss between the reconstructed image and the original image, the variational encoder, decoder, image encoder and softmax classifier in the initial pre-trained model are trained to obtain the target pre-trained model. Training the liveness detection model based on the first image, the restored image, and the predicted classification result includes: The restoration loss is determined based on the first image and the restored image; The classification loss is determined based on the predicted classification results; The liveness detection model is trained based on the recovery loss and the classification loss; The liveness detection model also includes a multi-loss fusion module, which trains the liveness detection model based on the restoration loss and the classification loss, including: The fusion loss is obtained by weighted summation of the restoration loss and the classification loss through the multi-loss fusion module. When the fusion loss is less than a preset value, the most recently determined liveness detection model is used as the trained liveness detection model. When the fusion loss is not less than the preset value, the parameters of the liveness detection model are adjusted based on the restoration loss and the classification loss, and the process returns to the step of obtaining the first image from the image sample set to continue training the liveness detection model.
2. The method according to claim 1, characterized in that, Determining the restoration loss based on the first image and the restored image includes: The restoration loss between the first image and the restored image is calculated using the mean square error method.
3. The method according to claim 1, wherein the blurring process for the first image is Gaussian blurring.
4. A method for detecting liveness, characterized in that, The method includes: The image to be detected is restored by the image restoration module in the trained liveness detection model to obtain the restored image; the trained liveness model is obtained by the training method of the liveness detection model according to any one of claims 1-3; The restored image is used to extract features through the feature extraction module in the trained liveness detection model to obtain the corresponding facial feature information. The training liveness detection model classifies the input facial feature information using its classification module to obtain a detection result. The detection result is used to indicate whether the object corresponding to the image to be detected is a real person or a non-real person. The initial parameters of the feature extraction module are obtained through pre-training. The parameters of the feature extraction module are pre-trained to obtain the initial parameters of the feature extraction module, including: The initial pre-trained model is trained using an unlabeled image sample set to obtain the target pre-trained model; The parameters of the encoding unit in the target pre-trained model are used as the initial parameters of the feature extraction module, wherein the encoding unit has the same network structure as the feature extraction module; The step of training the initial pre-trained model using an unlabeled image sample set to obtain the target pre-trained model includes: The original images in the unlabeled image sample set are divided into multiple image blocks; The multiple image blocks are processed through the first processing branch: each image block is encoded using a variational encoder to obtain a visual marker corresponding to each image block; then the visual markers are decoded using a decoder to obtain a reconstructed image of the original image; The second processing branch processes the multiple image patches: a portion of the multiple image patches is masked to obtain a third image containing the masked image patches; each image patch in the third image is mapped into a vector and flattened into a two-dimensional matrix, which is then input into an image encoder to generate an image feature vector; the image feature vector is then output through a softmax classifier to predict the visual label corresponding to the masked image patch. Based on the softmax loss between the visual markers predicted by the second processing branch and the visual markers corresponding to the occluded image patches in the first processing branch, and the mean square error loss between the reconstructed image and the original image, the variational encoder, decoder, image encoder, and softmax classifier in the initial pre-trained model are trained to obtain the target pre-trained model.
5. A training device for a liveness detection model, characterized in that, The liveness detection model includes an image restoration module, a feature extraction module, and a classification module. The classification module, composed of fully connected layers, is used to determine whether the input information corresponds to a real person or a non-real person. The device includes: A pre-training unit is used to pre-train the parameters of the feature extraction module to obtain the initial parameters of the feature extraction module; The acquisition unit is used to acquire a first image from an image sample set, and a second image of the first image after blurring. The restoration unit is used to perform restoration processing on the second image through the image restoration module to obtain a restored image; The feature extraction unit is used to extract features from the restored image through the feature extraction module to obtain corresponding facial feature information; A classification unit is used to classify the input facial feature information through the classification module to obtain a predicted classification result; The training unit is used to train the liveness detection model based on the first image, the restored image, and the predicted classification result to obtain a trained liveness detection model. The pre-training unit is specifically used to: train the initial pre-training model using an unlabeled image sample set to obtain the target pre-training model; The parameters of the encoding unit in the target pre-trained model are used as the initial parameters of the feature extraction module, wherein the encoding unit has the same network structure as the feature extraction module; The pre-training unit is specifically used to: divide the original images in the unlabeled image sample set into multiple image blocks; The multiple image blocks are processed through the first processing branch: each image block is encoded using a variational encoder to obtain a visual marker corresponding to each image block; then the visual markers are decoded using a decoder to obtain a reconstructed image of the original image; The second processing branch processes the multiple image patches: a portion of the multiple image patches is masked to obtain a third image containing the masked image patches; each image patch in the third image is mapped into a vector and flattened into a two-dimensional matrix, which is then input into an image encoder to generate an image feature vector; the image feature vector is then output through a softmax classifier to predict the visual label corresponding to the masked image patch. Based on the softmax loss between the visual markers predicted by the second processing branch and the visual markers corresponding to the occluded image blocks in the first processing branch, and the mean square error loss between the reconstructed image and the original image, the variational encoder, decoder, image encoder and softmax classifier in the initial pre-trained model are trained to obtain the target pre-trained model. The training unit is specifically used to determine the restoration loss based on the first image and the restored image; determine the classification loss based on the predicted classification result; and train the liveness detection model based on the restoration loss and the classification loss. The training unit is further configured to perform a weighted summation of the restoration loss and the classification loss using a multi-loss fusion module to obtain a fusion loss; when the fusion loss is less than a preset value, the most recently determined liveness detection model is used as the trained liveness detection model; when the fusion loss is not less than the preset value, the parameters of the liveness detection model are adjusted based on the restoration loss and the classification loss, and the unit returns to the step of obtaining the first image from the image sample set to continue training the liveness detection model.
6. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1-4 by executing the executable instructions.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-4.