Living body detection method and device, electronic device and storage medium

Abstract

The invention relates to a living body detection method and device, an electronic device and a storage medium, and is applied to the field of target recognition, and the method comprises the steps: inputting a to-be-detected image collected by monocular equipment into a target lightweight network structure in a living body detection network, and obtaining a face detection frame and a living body label corresponding to the face detection frame; and the target lightweight network structure is used for acquiring target global features, including dense semantic information in the to-be-detected image, in the to-be-detected image. Cutting a human face detection frame corresponding to the target living body label containing the living body content to obtain a cut human face, and inputting the cut human face into the target texture discrimination network to obtain a living body detection result; and the target texture judgment network is used for carrying out classification judgment on the clipped face on a texture feature level. According to the method and the device, the problem that hardware cost saving and accurate face detection effect keeping cannot be taken into consideration under the condition that only one-way color image input exists in the prior art is solved.

Description

Technical Field

[0001] This application relates to the field of target recognition, and in particular to a method, apparatus, electronic device, and storage medium for liveness detection. Background Technology

[0002] Facial recognition and authentication technologies provide a convenient channel for daily identity verification. However, to ensure the security of personal information during authentication, facial forgery detection technology has become a necessary step before facial recognition. In facial forgery detection, additional hardware devices such as multi-view cameras, depth cameras, or thermal infrared cameras are used to effectively supplement the basic color image input, thus overcoming the inherent limitations of purely visual solutions in terms of perception.

[0003] However, using additional hardware devices such as multi-view cameras, depth cameras, or thermal infrared cameras will increase the additional hardware cost. Moreover, in actual identity verification, due to different scenarios, equipment, and computing power, especially when facing a large number of gate card devices that may have multiple live subjects or identity verification requirements from mobile devices, using additional camera hardware devices cannot maintain accurate face detection results when there is only a single color image input.

[0004] There is currently no effective solution to the technical problem that it is impossible to balance saving hardware costs and maintaining accurate face detection results when only a single color image input is available. Summary of the Invention

[0005] This embodiment provides a liveness detection method, apparatus, electronic device, and storage medium to address the problem in related technologies that it is impossible to balance saving hardware costs and maintaining accurate face detection results when only a single-channel color image input is available.

[0006] Firstly, this embodiment provides a liveness detection method, the method comprising:

[0007] Acquire the image to be detected from the monocular device;

[0008] The image to be detected is input into the target lightweight network structure in the liveness detection network to obtain face detection boxes and corresponding liveness labels; the target lightweight network structure is used to obtain target global features in the image to be detected; the target global features contain dense semantic information in the image to be detected;

[0009] The face detection bounding box corresponding to the target liveness label is cropped to obtain the cropped face; the cropped face is then input into the target texture discrimination network to obtain the liveness detection result; the target liveness label is a label containing liveness content; the target texture discrimination network is trained by a texture feature network and is used to classify and judge the cropped face at the texture feature level.

[0010] In some embodiments, the liveness detection network further includes a self-supervised image feature model, which is used to acquire dense semantic features in the image; the method further includes:

[0011] Image enhancement processing is performed on the training images acquired by the monocular device;

[0012] The training image after image enhancement is input into the self-supervised image feature model in the liveness detection network to train the self-supervised image feature model and obtain the trained target self-supervised image feature model.

[0013] The training images acquired by the monocular device are subjected to image enhancement and scaling processing;

[0014] The training image after image enhancement and scaling is fed into a lightweight network structure, which is then trained to obtain the trained target lightweight network structure. The target lightweight network structure acquires global features of the target in the image to be detected using the target self-supervised image feature model, while employing dense semantic features from the training image to constrain these global features. The lightweight network structure is used to acquire global features from the training image.

[0015] In some embodiments, the step of using dense semantic features in the training image to constrain the global features includes:

[0016] The dense semantic features and global features in the training image are respectively divided into hierarchical features to obtain multiple dense semantic sub-features and multiple global sub-features of the training image;

[0017] Based on a preset cosine loss function, the multiple dense semantic sub-features are subjected to feature constraints in a hierarchical structure, and the target global sub-features are output.

[0018] In some embodiments, the liveness detection network further includes a linear network layer;

[0019] The step of training the lightweight network structure to obtain the trained target lightweight network structure includes:

[0020] The target global sub-features are input into the linear network layer of the liveness detection network to obtain a face prediction bounding box including the face and the predicted liveness label corresponding to the face prediction bounding box.

[0021] Calculate the face bounding box localization loss between the predicted face bounding box and the labeled face bounding box in the training image, and the label category loss between the predicted liveness label in the predicted face bounding box and the real liveness label in the training image;

[0022] Based on the face bounding box localization loss and the label category loss, the lightweight network structure in the liveness detection network is trained to obtain the target lightweight network structure.

[0023] In some embodiments, the step of training the lightweight network structure in the liveness detection network based on the face bounding box localization loss and the label category loss to obtain the target lightweight network structure includes:

[0024] A bipartite graph matching operation is performed on the labeled face bounding boxes and the face prediction bounding boxes in the training image to obtain face prediction bounding boxes that match the labeled face bounding boxes in the training image, and the predicted liveness labels of face prediction bounding boxes that do not match the preset labeled face bounding boxes are set to empty.

[0025] Based on the labeled face bounding boxes in the training images and the face prediction boxes that match the preset labeled face bounding boxes, calculate the face bounding box localization loss in the liveness detection network;

[0026] The label category loss in the liveness detection network is calculated based on the predicted liveness label in the face prediction bounding box that matches the preset labeled face, and the real liveness label in the labeled face bounding box in the training image.

[0027] Based on the face bounding box localization loss, the label category loss, and the preset cosine loss function, the lightweight network structure in the liveness detection network is trained to obtain the target lightweight network structure.

[0028] In some embodiments, the method further includes:

[0029] Determine whether the predicted liveness label corresponding to the face prediction bounding box including the face is the target label category; the target label category represents the label category corresponding to the face in the training image being a live object;

[0030] If so, the face image in the face prediction box is obtained, and the face image is enhanced and scaled to obtain the processed face image;

[0031] The processed face image is input into a preset texture feature network to obtain the predicted label category;

[0032] Based on multiple preset liveness labels and the category probabilities corresponding to the predicted label categories, the texture feature network is trained using loss to obtain the target texture discrimination network.

[0033] In some embodiments, inputting the image to be detected into the target lightweight network structure of the liveness detection network includes:

[0034] The image to be detected is scaled to obtain a scaled image to be detected; the image to be detected is a complete original-size image acquired by the monocular device.

[0035] The scaled image to be detected is input into the lightweight network structure of the liveness detection network.

[0036] Secondly, this embodiment provides a liveness detection device, which includes: an acquisition module, a processing module, and a detection module;

[0037] The acquisition module is used to acquire the image to be detected collected by the monocular device;

[0038] The processing module is used to input the image to be detected into the target lightweight network structure in the liveness detection network to obtain a face detection box and a liveness label corresponding to the face detection box; the target lightweight network structure is used to obtain target global features in the image to be detected; the target global features contain dense semantic information in the image to be detected;

[0039] The detection module is used to crop the face detection box corresponding to the target liveness label to obtain the cropped face; and input the cropped face into the target texture discrimination network to obtain the liveness detection result; the target liveness label is a label containing liveness content; the target texture discrimination network is trained by a texture feature network and is used to classify and judge the cropped face at the texture feature level.

[0040] Thirdly, this embodiment provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the liveness detection method described in the first aspect above.

[0041] Fourthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the liveness detection method described in the first aspect above.

[0042] Compared with related technologies, the liveness detection method, apparatus, electronic device, and storage medium provided in this embodiment acquire a complete image to be detected using a monocular device, and then perform full-image liveness detection on the image to be detected without additional hardware. The image to be detected is input into a target lightweight network structure to obtain face detection boxes and corresponding liveness labels. During training, the target lightweight network is constrained based on a target self-supervised image feature model, utilizing powerful dense semantic extraction capabilities to enhance the semantic feature extraction ability of the lightweight network. This improves the feature extraction capability in the image without increasing computational power consumption, laying the foundation for improving the accuracy of face forgery detection. Simultaneously, combined with a target texture discrimination network, the face detection boxes output from the target lightweight network structure and their corresponding liveness labels are identified at the texture feature level, improving the accuracy of texture forgery detection.

[0043] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0044] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0045] Figure 1 This is a hardware structure block diagram of the terminal of the liveness detection method provided in the embodiments of this application;

[0046] Figure 2 This is a flowchart of the liveness detection method provided in the embodiments of this application;

[0047] Figure 3 This is a flowchart of the full-image semantic awareness liveness detection method provided in this specific embodiment;

[0048] Figure 4 This is a schematic diagram of the lightweight liveness detection network provided in this specific embodiment;

[0049] Figure 5 This is a schematic diagram of the texture discrimination network provided in this specific embodiment;

[0050] Figure 6 This is a structural block diagram of the liveness detection device provided in the embodiments of this application. Detailed Implementation

[0051] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0052] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0053] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the liveness detection method provided in this application embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0054] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the liveness detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0055] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0056] Currently, various methods exist for liveness detection. One approach involves using a binocular camera to generate a depth map from color and infrared images. The acquired color image, infrared image, and generated depth map are then fed into a liveness detection model with a Vision Transformer (VIT) structure. Finally, features from multiple modalities are interactively fused to obtain the final liveness detection result. This approach utilizes data from multiple modalities to extract effective classification features.

[0057] However, this liveness detection method, after acquiring color and infrared images, still needs to use these images in conjunction with camera intrinsic parameters to calculate a depth map, which faces several challenges: First, it requires hardware support for binocular cameras. If the equipment to be deployed is inexpensive but does not support binocular cameras, deployment will be impossible. Second, using camera intrinsic parameters in conjunction with color and infrared images for calculation requires camera calibration, which implicitly places higher demands on the processing precision of the equipment. Third, calculating depth images using color and infrared images consumes a lot of computing power, thus greatly increasing the time required for the entire liveness detection process. Fourth, the depth map generated by binocular calibration is highly dependent on ambient light. When the ambient light is too dark or too bright, the generated depth image will no longer be reliable. Even in normal scenes, the depth map generated by binocular calibration contains a large number of outliers, which will have a significant impact on the liveness detection results.

[0058] Another method for liveness detection involves acquiring a single-channel input image, using a face detection module to detect faces in the input image, and cropping the detected faces at three different scales. These three scaled images are then input into three different neural networks for liveness detection. If all three predictions exceed a threshold, the detected face is considered a live face. This method requires no user interaction and does not rely on specific hardware devices.

[0059] However, this liveness detection method requires cropping the detected faces to different sizes before feeding the cropped images into the liveness detection model. However, the cropping ratios for positive and negative samples are inconsistent. In real-world applications, without corresponding liveness labels and only having face bounding boxes, a fixed cropping scheme must be used. This imbalance in the training data will negatively impact the model's final performance. Furthermore, in scenarios involving multiple people, especially when both live and non-live individuals are present, using an outward cropping method will likely result in live and non-live individuals being cropped together, making it difficult for the model to determine whether the target in the image is live or not, leading to misclassification.

[0060] Therefore, in order to perform efficient and accurate liveness detection on a monocular device without incurring additional hardware costs or other calibration methods, and to correctly handle the liveness detection of multiple faces of similar size and location in the screen in scenarios such as turnstiles and card readers where multiple people may be present, this embodiment provides a liveness detection method.

[0061] Figure 2 This is a flowchart of the liveness detection method provided in the embodiments of this application, such as... Figure 2 As shown, the process includes the following steps:

[0062] Step S210: Obtain the image to be detected acquired by the monocular device; input the image to be detected into the target lightweight network structure in the liveness detection network to obtain the face detection box and the corresponding liveness label; the target lightweight network structure is used to obtain the target global features in the image to be detected; the target global features contain dense semantic information in the image to be detected.

[0063] The process begins by collecting the image to be detected from a monocular device. This image is a complete, original-size image acquired by the monocular device and can be a color image or an infrared image. When inputting the image to be detected into the target lightweight network structure of the liveness detection network, the image to be detected needs to be scaled first to obtain a scaled image. Then, the scaled image to be detected is input into the lightweight network structure of the liveness detection network.

[0064] Here, when scaling the image to be detected, the scaling is mainly determined based on the size of the original input image and the computing power of the device currently processing the liveness detection method. Depending on the application scenario, the original input image to be detected is scaled to different sizes, such as 640×640 and 320×320, etc. The scaling result of the image to be detected is not limited here.

[0065] After scaling the image to be detected, the scaled image is input into the target lightweight network structure of the trained liveness detection network to obtain the predicted face detection boxes and the corresponding liveness labels. The target lightweight network structure is trained from a lightweight network structure, using lightweight network architectures such as MobileNet and ShuffleNet, to achieve fast and efficient feature extraction on devices with limited computing power, further improving the efficiency of subsequent liveness detection.

[0066] Meanwhile, the target lightweight network structure acquires the target global features in the image to be detected. However, the target global features also need to include the dense semantic features of the image to be detected, so as to enable efficient and accurate feature extraction of the image to be detected on devices with limited computing power. Among them, the dense semantic features of the image refer to assigning a semantically rich vector representation to each pixel or very dense sampling point in the image; this vector not only encodes the visual appearance of the pixel, such as color or texture, but also encodes its semantic category, such as "person", "vehicle", "sky", "road" and its contextual relationship.

[0067] The liveness detection network in this embodiment also includes a self-supervised image feature model. This model is used to acquire dense semantic features from images. Specifically, it involves performing image enhancement processing on training images acquired by a monocular device; inputting the enhanced training images into the self-supervised image feature model within the liveness detection network to train the model, resulting in a trained target self-supervised image feature model. Through this trained target self-supervised model, the dense semantic features of the image can be learned more accurately.

[0068] Meanwhile, a lightweight network structure is used to acquire global features from the training images. In training the lightweight network structure to obtain the target lightweight network structure, the training images acquired by the monocular device are first subjected to image enhancement and scaling. Then, the enhanced and scaled training images are input into the lightweight network structure for training, resulting in the trained target lightweight network structure.

[0069] Specifically, through the target self-supervised image feature model, in the process of obtaining the target global features in the image to be detected by the target lightweight network structure, the dense semantic features in the training image are used to constrain the global features; so that the trained target lightweight network structure can also obtain the dense semantic features of the image.

[0070] For example, the image enhancement operations here are only used during the training phase of the self-supervised image feature model and lightweight network structure. They can be operations such as adding image noise, adjusting image color, random cropping, and flipping, depending on the different application scenarios. No specific operation for image enhancement is limited here.

[0071] By utilizing the dense semantic features of images obtained from a self-supervised image feature model during the training phase, feature constraints are applied to the global features of the lightweight network structure. This enables the trained target lightweight network structure to acquire global features rich in dense semantic information from the image, thereby reducing the computational cost and parameter size during feature extraction.

[0072] Step S220: Crop the face detection box corresponding to the target liveness label to obtain the cropped face; and input the cropped face into the target texture discrimination network to obtain the liveness detection result; the target liveness label is a label containing liveness content; the target texture discrimination network is trained by the texture feature network and is used to classify and judge the cropped face at the texture feature level.

[0073] The liveness detection network also includes a linear network layer. After the scaled image to be detected is input into the lightweight network structure of the liveness detection network, a target global feature rich in dense semantic features of the image to be detected is obtained. Then, the target global feature is input into the linear network layer, and while performing face target detection to obtain face detection boxes, the liveness result is also judged, that is, the face detection box and the corresponding liveness label are predicted.

[0074] Subsequently, since the face detection box contains background and other content in addition to the detected target face, it is necessary to crop the face detection box in the original input image to obtain the cropped face. The cropped face may be a live face, or it may be a face photo with spoofing tendencies, a 3D / 2D mask, or other face.

[0075] For highly detailed and realistic 3D mask faces, the human eye often struggles to distinguish them from fakes. Therefore, this embodiment adds an additional texture feature model to determine the faces within the face detection bounding box at the level of detailed texture. A target texture discrimination network is obtained by training the texture feature network, and then the cropped face is used for liveness detection based on this network to output the liveness detection result.

[0076] Through the above steps, a complete image to be detected is acquired using a monocular device, and then full-image liveness detection is performed on the image without the need for additional hardware. The image to be detected is input into a target lightweight network structure to obtain face detection boxes and corresponding liveness labels. During training, the target lightweight network is constrained based on a target self-supervised image feature model, leveraging powerful dense semantic extraction capabilities to enhance the semantic feature extraction ability of the lightweight network. This improves the feature extraction capability in the image without increasing computational power consumption, laying the foundation for improving the accuracy of face forgery detection. Simultaneously, combined with a target texture discrimination network, texture feature recognition is performed on the face detection boxes and corresponding liveness labels output from the target lightweight network structure, improving the accuracy of texture forgery detection.

[0077] Since the input to the liveness detection network is the complete original input image without any cropping, it is not affected by different cropping ratios when cropping faces. Even if there are multiple faces in the cropped area, it can correctly distinguish the liveness results of different faces, and will not have the situation where a single image contains both live and non-live faces due to large-scale cropping. Furthermore, it can avoid the problem of incorrectly distinguishing between live and non-live faces when faces are close together because the cropped input of the liveness detection network contains both live and non-live faces.

[0078] In some embodiments, when training the target lightweight network structure, step S210 uses dense semantic features in the training image to constrain global features, including: performing hierarchical feature partitioning on the dense semantic features and global features in the training image to obtain multiple dense semantic sub-features and multiple global sub-features of the training image; based on a preset cosine loss function, constraining multiple global sub-features according to the hierarchical structure based on the multiple dense semantic sub-features, and outputting the target global sub-features.

[0079] During training, training images acquired by a monocular device are input into the liveness detection network. Global features are extracted from the training images via a lightweight network structure, and dense semantic features are extracted from the training images via a self-supervised image feature model. Since the self-supervised image feature model can extract high-quality dense semantic features from the input training images, it can provide positive scene semantic information for liveness detection, thereby improving the accuracy of identifying fake faces. However, the self-supervised image feature model has a large number of parameters and a large model size. Directly using this model for liveness detection consumes a lot of computing power, making it difficult to implement in most liveness recognition scenarios.

[0080] Therefore, in this embodiment, the dense semantic features extracted from the training images by the self-supervised image feature model are used to constrain the global features extracted from the training images by the lightweight network structure, so that the global features learned in the lightweight network structure are as close as possible to the dense semantic features extracted by the self-supervised image feature model. Specifically, the structures of the lightweight network structure and the self-supervised image feature model are divided into multiple feature extraction stages, and then multiple dense semantic sub-features and multiple global sub-features of the training image are extracted sequentially from the multiple feature extraction stages.

[0081] Based on different feature extraction stages, multiple dense semantic sub-features are used to constrain multiple global sub-features to obtain the target global sub-features for the final feature extraction stage. Specifically, a preset cosine loss function is used for feature constraint, ensuring that the global features output by the trained lightweight network structure contain dense semantic features. This avoids the problem of high parameter count and computational cost caused by using a self-supervised image feature model when extracting features from the image to be detected.

[0082] By extracting dense semantic features from training images through a self-supervised image feature model, feature constraints are applied to the global features extracted from the training images by the lightweight network structure, so that the global features learned in the lightweight network structure contain the dense semantic features of the training images. Then, the lightweight network structure is trained to obtain the target lightweight network structure.

[0083] The method for training the lightweight network structure includes: inputting the target global sub-features into the linear network layer of the liveness detection network to obtain face prediction boxes including the face and the corresponding predicted liveness labels; calculating the face box localization loss between the face prediction boxes and the labeled face boxes in the training image, and the label category loss between the predicted liveness labels in the face prediction boxes and the real liveness labels in the training image; and training the lightweight network structure in the liveness detection network based on the face box localization loss and the label category loss to obtain the target lightweight network structure.

[0084] In the lightweight network structure training, the target global sub-features constrained by the dense semantic features in the self-supervised image feature model are first input into two linear network layers, and then face prediction boxes including faces and corresponding liveness labels are obtained.

[0085] A matching operation is performed between the predicted face bounding boxes and the labeled face bounding boxes in the training image to obtain successfully matched face bounding boxes, and the predicted liveness label category of the successfully matched face bounding boxes is determined; for face bounding boxes that fail to match, their corresponding predicted liveness label category is set to "empty". Subsequently, the face bounding box localization loss between the successfully matched face bounding boxes and the labeled face bounding boxes in the training image is calculated, and the label category loss between the predicted liveness label in the successfully matched face bounding boxes and the true liveness label in the training image is calculated.

[0086] Furthermore, the lightweight network structure in the liveness detection network is trained using face bounding box localization loss and label category loss to obtain the target lightweight network structure. This includes: performing bipartite graph matching on the labeled face bounding boxes and predicted face bounding boxes in the training image to obtain predicted face bounding boxes that match the labeled face bounding boxes in the training image, and setting the predicted liveness labels of the predicted face bounding boxes that do not match the preset labeled face bounding boxes to empty; calculating the face bounding box localization loss in the liveness detection network based on the labeled face bounding boxes in the training image and the predicted face bounding boxes that match the preset labeled face bounding boxes; calculating the label category loss in the liveness detection network based on the predicted liveness labels in the predicted face bounding boxes that match the preset labeled face bounding boxes and the real liveness labels in the labeled face bounding boxes in the training image; and training the lightweight network structure in the liveness detection network using the face bounding box localization loss, label category loss, and a preset cosine loss function to obtain the target lightweight network structure.

[0087] Based on the face bounding box localization loss and label category loss, a loss function is obtained by training the lightweight network structure in the liveness detection network. Then, based on this loss function, the lightweight network structure in the liveness detection network is trained again to obtain the target lightweight network structure. By using the target lightweight network structure, liveness detection is performed simultaneously with face detection, which helps improve the accuracy of liveness detection.

[0088] In some embodiments, the liveness detection method further includes: determining whether the predicted liveness label corresponding to the face prediction bounding box including the face is a target label category; the target label category indicates that the face in the training image is a label category corresponding to liveness; if so, acquiring the face image in the face prediction bounding box, and performing image enhancement and scaling processing on the face image to obtain a processed face image; inputting the processed face image into a preset texture feature network to obtain the predicted label category; and performing loss training on the texture feature network according to multiple preset liveness labels and the category probabilities corresponding to the predicted label categories to obtain a target texture discrimination network.

[0089] Specifically, after outputting the face prediction bounding box and the corresponding predicted liveness label through a lightweight network structure, it is determined whether the predicted liveness label corresponding to the current face prediction bounding box is the target label category, such as whether it is a category containing live faces other than empty categories or non-liveness categories.

[0090] If the predicted liveness label corresponding to the current face prediction bounding box is the target label category, then the face prediction bounding box needs to be cropped from the original input training image to obtain the face image within the face prediction bounding box, and then image enhancement and scaling operations are performed to obtain the processed face image.

[0091] Subsequently, the processed face image is input into a preset texture feature network, i.e., a texture classification model, to classify and judge the texture in the human image and obtain the predicted label category. The texture feature network classifies the face image according to the texture features of the object surface in the image, such as regularity, directionality, roughness, etc.

[0092] The predicted label category output by the texture feature network is combined with the category probability corresponding to the predicted label category and multiple liveness label categories to train the texture feature network using loss, thus obtaining the target texture discrimination network.

[0093] The present embodiment will be described and explained below through specific examples.

[0094] In response to the need for large-scale deployment of gate card-swiping devices or facial recognition authentication from mobile devices, this specific embodiment provides a full-image semantic perception liveness detection method, which can accurately handle the liveness detection of multiple faces with similar positions and sizes in the image in scenarios such as gate card-swiping machines where multiple people may appear.

[0095] Figure 3 This is a flowchart of the full-image semantic awareness liveness detection method provided in this specific embodiment. (Reference) Figure 3 First, images are collected from a monocular device. Taking a gate card reader as an example, these images can be color or infrared. For the collected images, there's no need to use an additional detection module for face bounding box detection; instead, the images are directly input into a lightweight liveness detection network.

[0096] For lightweight liveness detection networks, the input image is preprocessed first. During the training phase of the liveness detection network, the image preprocessing is divided into two parts: first, image enhancement is performed on the input image, such as adding image noise, adjusting image colors, random cropping, or flipping. Image enhancement is only used during the training phase; it is not required during the actual application inference phase.

[0097] Figure 4 This is a schematic diagram of the lightweight liveness detection network provided in this specific embodiment. (Reference) Figure 4 Here, we take the DINOv3 model as an example of a self-supervised image feature model. The lightweight liveness detection network includes a lightweight network structure and the DINOv3 model. After inputting the image into the lightweight liveness detection network and performing enhancement operations on the input image, no scaling operation is performed on the network input of the DINOv3 model branch to minimize the impact of abnormal image deformation on feature extraction and to retain as much original information of the input image as possible. However, for the input of the lightweight network structure branch, all input images need to be scaled to a uniform size before network training.

[0098] In the application reasoning stage of practical applications, it is only necessary to perform scaling operations on the input image. When performing scaling operations, the original input image can be scaled to different sizes, such as 640×640, 320×320, etc., depending on the size of the original input image and the computing power of the device.

[0099] During the training phase of the lightweight liveness detection network, the enhanced images are fed into the pre-trained DINOv3 network. DINOv3 is a self-supervised visual semantic feature extraction model pre-trained from hundreds of millions of image data. This model can extract high-quality dense semantic features from the input images. The dense semantic features extracted by DINOv3 can provide positive scene semantic information for liveness detection, enabling the model to fully detect forgery clues of fake faces.

[0100] Leveraging DINOv3's powerful dense semantic feature understanding capabilities, it can fully utilize the background and semantic information in the original input image to more effectively extract clues about face forgery, thus eliminating the need for additional hardware devices such as binocular modules, depth modules, etc., saving costs and resources.

[0101] However, DINOv3 has a large number of parameters and a large model size, making direct inference using DINOv3 difficult to implement given the computing power of devices in most liveness detection scenarios. Therefore, this specific embodiment employs a lightweight network structure, such as MobileNet or ShuffleNet, to achieve fast and efficient feature extraction on devices with limited computing power. The input to this lightweight network differs from that of DINOv3; in addition to image enhancement, it requires further scaling operations to enable more efficient processing on the device side.

[0102] For both DINOv3 and the lightweight network, the entire structure is divided into four stages. The dense semantic sub-features extracted from each stage of DINOv3 are named Fd1, Fd2, Fd3, and Fd4, while the global sub-features extracted from the lightweight network are named Fs1, Fs2, Fs3, and Fs4. Then, the features extracted from DINOv3 (Fd1, Fd2, Fd3, and Fd4) are used to constrain the global features Fs1, Fs2, Fs3, and Fs4 extracted from the lightweight network. This ensures that the features learned from the scaled lightweight network are as close as possible to the densely semantically rich features extracted by DINOv3 from the original large image. DINOv3 is only used during the training phase to extract densely semantically rich features to constrain the features extracted from the lightweight network; therefore, in actual application inference, the computationally expensive and parameter-intensive DINOv3 is not required.

[0103] Specifically, a cosine similarity constraint loss function is used, which constrains the global sub-features through dense semantic sub-features, and outputs the target global sub-features. This loss function... Expressed as a formula:

[0104] ;

[0105] Where N represents the four stages obtained by dividing the DINOv3 model and its lightweight network structure. This represents the dense semantic sub-features extracted from the DINOv3 network in the i-th stage. This represents the global sub-feature extracted from the lightweight network structure in the i-th stage.

[0106] The target global sub-feature Fs4, extracted from the lightweight network and constrained by the DINOv3 model, is fed into two linear network layers to predict face detection boxes and liveness labels, respectively, resulting in face prediction boxes and corresponding predicted liveness labels.

[0107] Meanwhile, before training the lightweight liveness detection network, a fixed network output size needs to be set, specifically defining the number of output face prediction bounding boxes and the number of predicted liveness label categories. For scenarios such as turnstiles and access control systems, the maximum number of faces in the detected face prediction bounding boxes can be limited to M=10, which can handle most scenarios without incurring excessive performance overhead. For other scenarios, the maximum number of detected faces can be freely set according to the characteristics of the scenario to meet the needs of different scenarios.

[0108] Setting the maximum number of faces M=10 means the network will always output 10 face prediction boxes and obtain predicted liveness labels for each of these 10 face boxes. When calculating the liveness label loss function, considering that there are usually fewer than 10 faces for the turnstile under normal circumstances, an additional empty category needs to be added in addition to the existing liveness label categories to indicate that there is no corresponding face in the face box of the actual image.

[0109] For example, if the liveness category label only has two categories, ["live", "non-live"], then during network estimation, it will become ["live", "non-live", "empty"]. If the liveness category label is divided into 4 categories according to the face liveness attack type, then when performing liveness prediction, an additional category, "empty", needs to be added to make it 5 categories.

[0110] Since the lightweight network predicts M face bounding boxes, and the number of labeled face bounding boxes varies across different input images in the training data, it is necessary to map the face bounding boxes predicted by the lightweight network to the labeled face bounding boxes. In this specific embodiment, the Hungarian algorithm is used to perform bipartite graph matching between the network-predicted face bounding boxes and the labeled face bounding boxes in the training images. For matched faces, the corresponding face category is the category in the label; for unmatched faces, the category label is "empty" (…). For the matched face prediction bounding boxes, the matching loss is calculated, specifically the matching category loss and the face bounding box localization loss. That is, based on the labeled face bounding boxes in the training image and the face prediction bounding boxes that match the preset labeled face bounding boxes, the face bounding box localization loss in the liveness detection network is calculated. Based on the predicted liveness labels in the face prediction bounding boxes that match the pre-defined labeled faces, and the real liveness labels in the labeled face bounding boxes in the training images, the label category loss in the liveness detection network is calculated. The matching loss can be expressed by the following formula:

[0111] ;

[0112] ;

[0113] Where M is the total number of faces output by the lightweight network structure, and i represents the i-th matched face target. For the network output target obtained by the i-th match, The true value of the category for the i-th matched face target is the real liveness label; This is the probability of the corresponding matching target category output by the network, i.e., the probability of predicting the category of the liveness label; This represents the ground truth value of the face bounding box of the i-th target, which is the labeled face bounding box. This represents the face bounding box corresponding to the matched target in the network output, i.e., the face prediction box. The weights of the IoU loss function represent the bounding box regression loss function used for object detection tasks. This represents the weights of the L1 loss function for the face bounding box. In one specific embodiment, =2, =5.

[0114] Combining the above methods, the comprehensive loss function of the multi-task network for face bounding box detection and liveness labeling is... It can be expressed by the following formula:

[0115] ;

[0116] Among them, face bounding box localization loss weight =1, the loss function for feature constraints in lightweight network structures. weight =0.1.

[0117] For sophisticated and realistic 3D mask attacks, an additional texture discrimination model is used for liveness detection. Only when the liveness detection network and the texture discrimination model both result in a liveness result is the final result considered live. This ensures that the entire solution has excellent anti-spoofing capabilities against various face forgery attacks, including highly sophisticated and realistic 3D mask attacks.

[0118] Meanwhile, the face bounding box also originates from the output of the liveness detection network. Even without a liveness detection module, most liveness detection methods typically require a face detection module to perform face detection, and then perform liveness determination based on the face detection results. Therefore, the scheme in this embodiment, which combines a liveness detection module with a texture discrimination model, does not introduce additional resource or time increases; and it achieves comprehensive and powerful defense against sophisticated texture forgery, screen copying attacks, paper printing attacks, and other attacks without the need for an additional face detection module.

[0119] While performing face detection, the system also determines the liveness detection result. Leveraging DINOv3's powerful dense semantic feature extraction capabilities, the lightweight network can accurately identify and judge non-liveness attack signals that are discernible to the human eye. However, for more sophisticated and realistic 3D mask attacks, the human eye often struggles to distinguish between real and fake. Therefore, an additional texture feature model, namely a texture discrimination network, is added to make judgments at the level of detailed texture.

[0120] Figure 5 This is a schematic diagram of the texture discrimination network provided in this specific embodiment. (Reference) Figure 5 First, the output of the lightweight network is used to determine whether the predicted face detection bounding box and its label category are liveness categories. If the label category is empty or not a liveness category, no further processing is required. If the label category is a liveness category, the face is cropped from the original input image based on its corresponding face bounding box. Then, image enhancement is performed, and the enhanced image is scaled, preferably to 224×224. The enhanced and scaled images are then input into a pure texture classification model. Using the cropped face image, more refined texture features are employed to determine 3D face liveness.

[0121] For the classification results obtained by the 3D mask texture discrimination network, binary cross-entropy is used as a constraint, and its loss function is... The calculation formula is as follows:

[0122] ;

[0123] in, This indicates the category result output by the network. This represents the probability of the corresponding category in the network output, where N represents the liveness label category.

[0124] Furthermore, a liveness detection result is output only if the liveness label corresponding to the face detection box output by the liveness detection network and the liveness detection result output by the texture discrimination network both indicate that the face in the current image is live; otherwise, a liveness detection result is output that the face in the current image is not live.

[0125] In the liveness detection network provided in this application embodiment, while outputting face bounding boxes for face target detection, it also outputs the liveness result corresponding to the liveness label category of the face bounding box, and constrains the output features extracted by the lightweight network using the DINOv3 model. Utilizing the powerful dense semantic feature extraction capability of DINOv3, the semantic feature extraction capability of the lightweight network is improved, enabling the lightweight network to understand the semantic information of each face in the image. Furthermore, since this liveness detection network uses the entire image as input, it can fully utilize the background information in the image. Combining facial semantic information and background information, it can more effectively extract face forgery information of other attack types besides detailed and realistic 3D masks, thereby greatly improving the accuracy of liveness detection.

[0126] It should be noted that the steps shown in the above process or in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions.

[0127] This embodiment also provides a liveness detection device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0128] Figure 6 This is a structural block diagram of the liveness detection device provided in the embodiments of this application, as shown below. Figure 6 As shown, the device includes: an acquisition module 10, a processing module 20, and a detection module 30.

[0129] The acquisition module 10 is used to acquire the image to be detected collected by the monocular device.

[0130] The processing module 20 is used to input the image to be detected into the target lightweight network structure in the liveness detection network to obtain the face detection box and the corresponding liveness label; the target lightweight network structure is used to obtain the target global features in the image to be detected; the target global features contain dense semantic information in the image to be detected.

[0131] The detection module 30 is used to crop the face detection box corresponding to the target liveness label to obtain the cropped face; and input the cropped face into the target texture discrimination network to obtain the liveness detection result; the target liveness label is a label containing liveness content; the target texture discrimination network is trained by the texture feature network and is used to classify and judge the cropped face at the texture feature level.

[0132] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0133] This embodiment also provides an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0134] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0135] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0136] S1, acquire the image to be detected from the monocular device; input the image to be detected into the target lightweight network structure in the liveness detection network to obtain the face detection box and the corresponding liveness label; the target lightweight network structure is used to obtain the target global features in the image to be detected; the target global features contain dense semantic information in the image to be detected.

[0137] S2, crop the face detection box corresponding to the target liveness label to obtain the cropped face; and input the cropped face into the target texture discrimination network to obtain the liveness detection result; the target liveness label is a label containing liveness content; the target texture discrimination network is trained by the texture feature network and is used to classify and judge the cropped face at the texture feature level.

[0138] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0139] Furthermore, in conjunction with the liveness detection methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the liveness detection methods described in the above embodiments.

[0140] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0141] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0142] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0143] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for detecting liveness, characterized in that, The method includes: Acquire the image to be detected from the monocular device; The image to be detected is input into the target lightweight network structure in the liveness detection network to obtain face detection boxes and corresponding liveness labels; the target lightweight network structure is used to obtain target global features in the image to be detected; the target global features contain dense semantic information in the image to be detected; The face detection bounding box corresponding to the target liveness label is cropped to obtain the cropped face; the cropped face is then input into the target texture discrimination network to obtain the liveness detection result; the target liveness label is a label containing liveness content; the target texture discrimination network is trained by a texture feature network and is used to classify and judge the cropped face at the texture feature level.

2. The liveness detection method according to claim 1, characterized in that, The liveness detection network further includes a self-supervised image feature model, which is used to acquire dense semantic features in the image; the method further includes: Image enhancement processing is performed on the training images acquired by the monocular device; The training image after image enhancement is input into the self-supervised image feature model in the liveness detection network to train the self-supervised image feature model and obtain the trained target self-supervised image feature model. The training images acquired by the monocular device are subjected to image enhancement and scaling processing; The training image after image enhancement and scaling is fed into a lightweight network structure, which is then trained to obtain the trained target lightweight network structure. The target lightweight network structure acquires global features of the target in the image to be detected using the target self-supervised image feature model, while employing dense semantic features from the training image to constrain these global features. The lightweight network structure is used to acquire global features from the training image.

3. The live detection method according to claim 2, characterized in that, The step of constraining the global features using dense semantic features in the training images includes: The dense semantic features and global features in the training image are respectively divided into hierarchical features to obtain multiple dense semantic sub-features and multiple global sub-features of the training image; Based on a preset cosine loss function, the multiple dense semantic sub-features are subjected to feature constraints in a hierarchical structure, and the target global sub-features are output.

4. The live detection method according to claim 3, characterized in that, The liveness detection network also includes a linear network layer; The step of training the lightweight network structure to obtain the trained target lightweight network structure includes: The target global sub-features are input into the linear network layer of the liveness detection network to obtain a face prediction bounding box including the face and the predicted liveness label corresponding to the face prediction bounding box. Calculate the face bounding box localization loss between the predicted face bounding box and the labeled face bounding box in the training image, and the label category loss between the predicted liveness label in the predicted face bounding box and the real liveness label in the training image; Based on the face bounding box localization loss and the label category loss, the lightweight network structure in the liveness detection network is trained to obtain the target lightweight network structure.

5. The liveness detection method according to claim 4, characterized in that, The step of training the lightweight network structure in the liveness detection network based on the face bounding box localization loss and the label category loss to obtain the target lightweight network structure includes: A bipartite graph matching operation is performed on the labeled face bounding boxes and the face prediction bounding boxes in the training image to obtain face prediction bounding boxes that match the labeled face bounding boxes in the training image, and the predicted liveness labels of face prediction bounding boxes that do not match the preset labeled face bounding boxes are set to empty. Based on the labeled face bounding boxes in the training images and the face prediction boxes that match the preset labeled face bounding boxes, calculate the face bounding box localization loss in the liveness detection network; The label category loss in the liveness detection network is calculated based on the predicted liveness label in the face prediction bounding box that matches the preset labeled face, and the real liveness label in the labeled face bounding box in the training image. Based on the face bounding box localization loss, the label category loss, and the preset cosine loss function, the lightweight network structure in the liveness detection network is trained to obtain the target lightweight network structure.

6. The liveness detection method according to claim 4, characterized in that, The method further includes: Determine whether the predicted liveness label corresponding to the face prediction bounding box including the face is the target label category; the target label category represents the label category corresponding to the face in the training image being a live object; If so, the face image in the face prediction box is obtained, and the face image is enhanced and scaled to obtain the processed face image; The processed face image is input into a preset texture feature network to obtain the predicted label category; Based on multiple preset liveness labels and the category probabilities corresponding to the predicted label categories, the texture feature network is trained using loss to obtain the target texture discrimination network.

7. The live detection method according to claim 1, characterized in that, The input of the image to be detected into the target lightweight network structure of the liveness detection network includes: The image to be detected is scaled to obtain a scaled image to be detected; the image to be detected is a complete original-size image acquired by the monocular device. The scaled image to be detected is input into the lightweight network structure of the liveness detection network.

8. A liveness detection device, characterized in that, The device includes: an acquisition module, a processing module, and a detection module; The acquisition module is used to acquire the image to be detected collected by the monocular device; The processing module is used to input the image to be detected into the target lightweight network structure in the liveness detection network to obtain a face detection box and a liveness label corresponding to the face detection box; the target lightweight network structure is used to obtain target global features in the image to be detected; the target global features contain dense semantic information in the image to be detected; The detection module is used to crop the face detection box corresponding to the target liveness label to obtain the cropped face; and input the cropped face into the target texture discrimination network to obtain the liveness detection result; the target liveness label is a label containing liveness content; the target texture discrimination network is trained by a texture feature network and is used to classify and judge the cropped face at the texture feature level.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the liveness detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the liveness detection method according to any one of claims 1 to 7.

Images