A face living body detection method and system based on machine learning
By adopting a shared encoder-multi-decoder head architecture and multi-task learning, combined with the supervision of reflectivity maps and LBP maps, the problem of insufficient generalization of end-to-end CNN liveness detection methods under complex lighting and attack conditions is solved, achieving high-precision and lightweight liveness detection results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIAN XITU ZHIGUANG INTELLIGENT TECH CO LTD
- Filing Date
- 2026-03-18
- Publication Date
- 2026-06-05
AI Technical Summary
Existing end-to-end CNN liveness detection methods have insufficient generalization performance under complex lighting, cross-device, and unknown attack conditions, making it difficult to effectively distinguish between real faces and fake materials, and the network structure is closed and has poor interpretability.
A shared encoder-multiple decoder head architecture is adopted, combined with the MiniFASNetV2 network, to extract and reconstruct features from near-infrared face images through multi-task learning. Reflectance maps and LBP maps are introduced as auxiliary supervision to achieve explicit constraints on illumination reflection patterns and texture structures, forming a set of multi-task prediction results. The model parameters are then optimized and updated through joint loss.
It significantly improves the model's generalization robustness under different lighting conditions and various attack scenarios, maintains discrimination accuracy, and achieves lightweight inference through model pruning, thereby improving the security and real-time performance of liveness detection.
Smart Images
Figure CN122157378A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and biometric security technology, and in particular to a face liveness detection method and system based on machine learning. Background Technology
[0002] With the widespread application of facial recognition technology in key scenarios such as mobile payment and access control security, its security protection issues have become increasingly prominent. To counter attacks using printed photos, video playback, and 3D masks, facial liveness detection has become a crucial means of preventing identity fraud. Near-infrared imaging, with its stability under low-light conditions and sensitivity to differences in reflectivity between skin and counterfeit materials, has gradually become an important technical approach in the field of liveness detection. Currently, most mainstream methods employ an end-to-end binary classification framework based on convolutional neural networks (CNNs). The acquired near-infrared facial images are directly input into a lightweight CNN network, and the classification layer outputs a "real" or "fake" result. These methods are simple in structure and highly computational, and have been widely used in commercial devices and mobile terminals.
[0003] However, existing end-to-end CNN-based liveness detection methods still have significant limitations. Because feature learning relies entirely on the final classification loss and lacks explicit physical constraints on reflectivity and texture patterns, the models are prone to learning pseudo-features associated with training samples, leading to insufficient generalization performance. Furthermore, the closed network structure and poor interpretability make it difficult to diagnose error sources and hinder the identification of subtle differences in material reflectivity and sophisticated counterfeit attacks. These issues cause traditional methods to experience a significant drop in liveness detection accuracy under complex lighting conditions, cross-device scenarios, and unknown attack conditions, failing to meet the robustness and reliability requirements of practical security applications.
[0004] Therefore, how to provide a face liveness detection method and system based on machine learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] One objective of this invention is to propose a face liveness detection method and system based on machine learning. This invention integrates near-infrared imaging and multi-task deep learning to construct a shared encoder-multi-decoder liveness detection model, which has the advantages of strong robustness, high discrimination accuracy and efficient inference.
[0006] A face liveness detection method based on machine learning according to an embodiment of the present invention includes the following steps: Near-infrared face images are acquired and labeled, and corresponding auxiliary supervision images are generated offline to form training sample units; A shared feature encoder with the MiniFASNetV2 network as the backbone is constructed, and a multi-task decoder head is built on it to form a shared encoder-multi-decoder head architecture. The training sample units are input into the shared encoder-multi-decoder head architecture to obtain classification output, reflectance map reconstruction output and LBP map reconstruction output, and generate a set of multi-task prediction results. Based on multi-task joint supervision, the joint loss weighted summation is performed on the multi-task prediction result set, and the parameters of the shared feature encoder are updated using a stochastic gradient descent optimizer to obtain the converged multi-task model. The converged multi-task model is pruned to obtain the inference model. The near-infrared face image to be tested is input into the inference model to obtain classification and generate a liveness detection result after post-processing.
[0007] Optionally, the generation of the training sample units specifically includes: Under near-infrared imaging conditions, facial images and their imaging metadata are acquired. The acquired near-infrared facial images are labeled as real or fake, and labels are established according to different presentation attack subclasses. A set of near-infrared facial images with real and fake labels and presentation attack subclass labels is output. The near-infrared face image set is processed for format unification and spatial alignment, including size standardization, grayscale intensity normalization and face region alignment, to generate a standardized near-infrared face image set; Based on the Retinex concept, the illumination component is obtained by convolving the face image with the Gaussian blur kernel. Then, the gray value of the standardized near-infrared face image corresponding to the face image is divided by the illumination component and a stabilization term is added to obtain a candidate set of reflectance maps. LBP images are extracted from each image in the standardized near-infrared face image set. Local binary patterns are calculated using a unified neighborhood and radius configuration. The grayscale difference between the center pixel and the neighboring pixels is compared and the result is encoded into a binary pattern to obtain local texture features, thus obtaining a candidate set of LBP images. Size alignment and naming alignment are performed on the candidate set of reflectance map and candidate set of LBP map. An index mapping table is built for each standardized near-infrared face image and its corresponding reflectance map and LBP map according to the correspondence. The training index list containing file path, true and false labels and attack subclass labels is output. The samples are divided according to the training index list. A hierarchical partitioning method is used to generate training subsets and validation subsets while maintaining the ratio of true to false and the distribution of attack subclasses. The hierarchical partitioning results are output. Standardized near-infrared face images, their corresponding reflectance maps and LBP maps, as well as true and false labels and presentation attack subclass labels from the training index list, are encapsulated into training sample units according to sample number.
[0008] Optionally, the generation of the shared encoder-multi-decoder head architecture specifically includes: A shared feature encoder is constructed using the MiniFASNetV2 network as the backbone. The MiniFASNetV2 network consists of an input layer, a convolutional normalized activation layer, four groups of depthwise separable convolutional modules and residual convolutional modules. Downsampling is achieved between each group through convolution with a stride of 2, and the output size is halved layer by layer. A classification head is built in the multi-task decoder head, which is composed of a global average pooling layer, a fully connected layer and a softmax layer connected in sequence. It is used to map the high-dimensional features output by the shared feature encoder to the probability output of true and false binary classification, and output the classification head initialization result. A reflectance map decoder is established in the multi-task decoder head, which consists of a multi-level upsampling layer, a convolutional fusion layer and a terminal convolutional layer. The reflectance distribution of the near-infrared face image is reconstructed by step-by-step upsampling and cross-layer stitching, and the initialization result of the reflectance map decoder is output. An LBP graph decoder is built in the multi-task decoder head, consisting of multi-level upsampling layers, convolutional fusion layers and terminal convolutional layers. The LBP texture features are reconstructed by feature upsampling at each level and cross-layer stitching, and the initialization result of the LBP graph decoder is output. The shared feature encoder, classification head, reflectance map decoder, and LBP map decoder are connected to form a shared encoder-multi-decoder head architecture. A callable interface for the shared encoder-multi-decoder head architecture is established according to a fixed order of classification head initialization results, reflectance map decoder initialization results, and LBP map decoder initialization results, and the network topology and parameter initialization results of the shared encoder-multi-decoder head architecture are output.
[0009] Optionally, the generation of the multi-task prediction result set specifically includes: Standardized near-infrared face images are read from training sample units and fed into the shared feature encoder in the shared encoder-multi-decoder head architecture. Convolution stacking, activation operations and scale compression are performed sequentially through multi-level encoding paths to form a multi-scale shared feature set. The multi-scale shared feature set is input into the classification head. First, global average pooling is performed on the fourth layer shared feature map to obtain the intermediate classification representation. Then, linear mapping is performed on the intermediate classification representation to obtain the classification output. The multi-scale shared feature set is input into the reflectance map decoder. Upsampling, feature concatenation and convolution fusion are performed on the multi-scale shared feature set step by step in a top-down path to obtain intermediate features for reflectance map reconstruction. Finally, terminal convolution is performed on the intermediate features for reflectance map reconstruction to generate the reflectance map reconstruction output. The multi-scale shared feature set is input into the LBP graph decoder. Upsampling, feature concatenation and convolution fusion are performed on the multi-scale shared feature set step by step in a top-down path to obtain the intermediate features of LBP graph reconstruction. Finally, terminal convolution is performed on the intermediate features of LBP graph reconstruction to generate the LBP graph reconstruction output. Spatial resolution and tensor channel alignment are performed on the classification output, reflectance map reconstruction output, and LBP map reconstruction output. Based on the output tensor order and spatial resolution alignment rules specified during network setup for the shared encoder-multiple decoder head architecture, the classification output is adjusted to match the classification labels in the training sample units. The reflectance map reconstruction output and LBP map reconstruction output are respectively adjusted to match the reflectance map and LBP map in the training sample units. Figure 1 The spatial dimensions and channel arrangement are aligned to form the aligned classification output, the aligned reflectance map reconstruction output, and the aligned LBP map reconstruction output; The aligned classification output, aligned reflectance map reconstruction output, and aligned LBP map reconstruction output are written into the multi-task prediction result set in a fixed order. The correspondence between these outputs and the training sample unit numbers is established and cached. The multi-task prediction result set is then output.
[0010] Optionally, the generation of the converged multi-task model specifically includes: Read the aligned classification output, aligned reflectance map reconstruction output, and aligned LBP map reconstruction output from the multi-task prediction result set, and read the corresponding classification label, reflectance map, and LBP map from the training sample unit, and establish corresponding supervision pairs of classification label, reflectance map, and LBP map according to the sample number; The cross-entropy loss of the main classification task is calculated based on the supervision of the classification label. The loss is accumulated item by item along the category dimension. The indicator value of the classification label in that category is multiplied by the negative of the logarithm of the probability of the classification output in that category. The summation is then applied to all categories to obtain the classification loss. Based on the reflectance map supervision, the pixel-level mean square error loss of the reflectance map reconstruction task is calculated. The total number of pixels is used as the normalization factor. The difference between the true value of each pixel position in the reflectance map and the predicted value of the aligned reflectance map reconstruction output at the same position is squared. The average of the squared differences of all pixel positions is then used to obtain the reflectance map reconstruction loss. The pixel-level mean squared error loss of the LBP image reconstruction task is calculated based on the LBP image supervision. The total number of pixels is used as the normalization factor. The difference between the true value at each position of the LBP image and the predicted value at the same position of the aligned LBP image reconstruction output is squared and averaged to obtain the LBP image reconstruction loss. The classification loss, reflectance map reconstruction loss, and LBP map reconstruction loss are weighted and summed together with preset weights to obtain the joint optimization objective. Backpropagation is performed on the shared encoder-multi-decoder head architecture based on the joint optimization objective. The gradient of the joint optimization objective with respect to the parameters of the shared feature encoder, the classification head, the reflectance map decoder, and the LBP map decoder is calculated. The above parameters are then updated synchronously using a stochastic gradient descent optimizer with momentum. When the convergence criterion, measured by the joint optimization objective, meets a preset threshold, the converged multi-task model is output.
[0011] Optionally, the generation of the inference model specifically includes: Load the converged multi-task model, read the parameters and network information of the shared feature encoder, classifier head, reflectance map decoder and LBP map decoder, complete the mapping and verification of weights and layer names according to the indexing rules of training sample units, and output a tailorable model description. Structural pruning is performed in the pruning model description, preserving the computational paths of the shared feature encoder and the classification head, removing all levels and parameters of the reflectance map decoder and LBP map decoder, and retaining only the main path structure with the input being a standardized near-infrared face image and the output being a classification, thus outputting a preliminary pruning model; The initial pruning model is subjected to structural optimization and parameter transfer. The parameters corresponding to the shared feature encoder and classifier head in the converged multi-task model are copied to the initial pruning model. The convolutional layer and the batch normalization layer are fused. The identity branch is folded with constants and redundant nodes are removed. The structurally optimized model is output. Perform inference equivalence verification on the structural optimization model, select the sample set of the verification subset to generate classifications using the converged multi-task model and the structural optimization model respectively, compare the class ranking and probability differences of the two on the same sample, and determine equivalence if it does not exceed the preset deviation threshold, and output the inference equivalence model. Using the inference equivalent model as the inference model, the input size, data type and output format are fixed, the input preprocessing requirements and output postprocessing order are recorded, and the inference model and runtime configuration are generated.
[0012] Optionally, the generation of the liveness detection result specifically includes: Receive the near-infrared face image to be tested, and perform size standardization, grayscale intensity normalization and face region alignment according to the runtime configuration to obtain a standardized near-infrared face image that meets the inference requirements; The standardized near-infrared face image is input into the inference model, and forward propagation is performed along a single path of the shared feature encoder and the classification head to output the classification result. The classification results are processed probabilistically, the probability distribution corresponding to each category is calculated, and the category with the highest probability is selected as the predicted category. The liveness detection result is then output. The liveness detection result is bound to the corresponding input image identifier to form an inference output record, which is then returned to the application interface.
[0013] A face liveness detection system based on machine learning according to an embodiment of the present invention includes: The near-infrared image acquisition and annotation module is used to acquire and annotate face images and imaging metadata, and generate corresponding reflectance maps and LBP maps offline. The shared encoder building block is used to build a shared feature encoder with the MiniFASNetV2 network as the backbone and build a multi-task decoder head on it. The multi-task prediction module is used to input training sample units into the shared encoder-multi-decoder head architecture and output classification results, reflectance map reconstruction results, and LBP map reconstruction results. The joint training optimization module is used to perform joint loss weighted summation on the multi-task prediction result set based on multi-task joint supervision and update the parameters of the shared feature encoder. The model pruning and inference generation module is used to perform structural pruning on the converged multi-task model to generate a lightweight inference model. The liveness detection module is used to input the near-infrared face image to be tested into the inference model and output the liveness detection result.
[0014] The beneficial effects of this invention are: This invention introduces a shared encoder-multi-decoder head architecture into near-infrared face liveness detection, achieving end-to-end optimization from feature learning objectives and training constraints to inference deployment. By using a shared feature encoder with a MiniFASNetV2 network as its backbone, and superimposing a classification head, reflectivity map decoder, and LBP map decoder in a multi-task decoding structure, this invention simultaneously supervises the classification output, reflectivity map reconstruction output, and LBP map reconstruction output during the training phase. This enables the model to perceive light reflection patterns and micro-texture structures while learning liveness detection features. This design explicitly introduces dual prior constraints on reflectivity and texture, significantly improving the model's generalization robustness under different lighting conditions and multi-class attack scenarios.
[0015] Furthermore, this invention employs a training-inference separation and computationally equivalent model pruning mechanism. After model convergence, it retains the shared feature encoder and classification head, removes auxiliary supervision branches, and performs parameter fusion and equivalence verification to form a lightweight inference model. This significantly reduces computational overhead while maintaining discriminative performance. This method balances the discriminative accuracy of multi-task learning with the real-time performance of embedded deployment. It can stably distinguish between real faces and highly realistic forgeries under near-infrared imaging conditions, improving the security, interpretability, and engineering application value of liveness detection systems. Attached Figure Description
[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a face liveness detection method based on machine learning proposed in this invention; Figure 2 This is a schematic diagram of a shared encoder-multi-decoder head architecture for a machine learning-based face liveness detection method proposed in this invention. Detailed Implementation
[0017] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0018] refer to Figures 1-2 A face liveness detection method based on machine learning includes the following steps: Near-infrared face images are acquired and labeled, and corresponding auxiliary supervision images are generated offline to form training sample units; A shared feature encoder with the MiniFASNetV2 network as the backbone is constructed, and a multi-task decoder head is built on it to form a shared encoder-multi-decoder head architecture. The training sample units are input into the shared encoder-multi-decoder head architecture to obtain classification output, reflectance map reconstruction output and LBP map reconstruction output, and generate a set of multi-task prediction results. Based on multi-task joint supervision, the joint loss weighted summation is performed on the multi-task prediction result set, and the parameters of the shared feature encoder are updated using a stochastic gradient descent optimizer to obtain the converged multi-task model. The converged multi-task model is pruned to obtain the inference model. The near-infrared face image to be tested is input into the inference model to obtain classification and generate a liveness detection result after post-processing.
[0019] In this embodiment, the generation of the training sample unit specifically includes: Under near-infrared imaging conditions, facial images and their imaging metadata are acquired. The acquired near-infrared facial images are labeled as real or fake, and labels are established according to different presentation attack subclasses. A set of near-infrared facial images with real and fake labels and presentation attack subclass labels is output. The near-infrared imaging conditions are imaging using an active light source of 850nm or 940nm, and the attack subclasses include printed photos, screen recordings, and 3D masks. The near-infrared face image set is processed for format unification and spatial alignment, including size standardization, grayscale intensity normalization and face region alignment, to generate a standardized near-infrared face image set; Based on the Retinex concept, the illumination component is obtained by convolving the face image with the Gaussian blur kernel. Then, the gray value of the standardized near-infrared face image corresponding to the face image is divided by the illumination component and a stabilization term is added to obtain a candidate set of reflectance maps. LBP images are extracted from each image in the standardized near-infrared face image set. Local binary patterns are calculated using a unified neighborhood and radius configuration. The grayscale difference between the center pixel and the neighboring pixels is compared and the result is encoded into a binary pattern to obtain local texture features, thus obtaining a candidate set of LBP images. Size alignment and naming alignment are performed on the candidate set of reflectance map and candidate set of LBP map. An index mapping table is built for each standardized near-infrared face image and its corresponding reflectance map and LBP map according to the correspondence. The training index list containing file path, true and false labels and attack subclass labels is output. The samples are divided according to the training index list. A hierarchical partitioning method is used to generate training subsets and validation subsets while maintaining the ratio of true to false and the distribution of attack subclasses. The hierarchical partitioning results are output. Standardized near-infrared face images, their corresponding reflectance maps and LBP maps, as well as true and false labels and presentation attack subclass labels from the training index list, are encapsulated into training sample units according to sample number.
[0020] In this embodiment, the generation of the shared encoder-multi-decoder head architecture specifically includes: A shared feature encoder is constructed using the MiniFASNetV2 network as the backbone. The MiniFASNetV2 network consists of an input layer, a convolutional normalized activation layer, four groups of depthwise separable convolutional modules and residual convolutional modules. Downsampling is achieved between each group through convolution with a stride of 2, and the output size is halved layer by layer. A classification head is built in the multi-task decoder head, which is composed of a global average pooling layer, a fully connected layer and a softmax layer connected in sequence. It is used to map the high-dimensional features output by the shared feature encoder to the probability output of true and false binary classification, and output the classification head initialization result. A reflectance map decoder is established in the multi-task decoder head, which consists of a multi-level upsampling layer, a convolutional fusion layer and a terminal convolutional layer. The reflectance distribution of the near-infrared face image is reconstructed by step-by-step upsampling and cross-layer stitching, and the initialization result of the reflectance map decoder is output. An LBP graph decoder is built in the multi-task decoder head, consisting of multi-level upsampling layers, convolutional fusion layers and terminal convolutional layers. The LBP texture features are reconstructed by feature upsampling at each level and cross-layer stitching, and the initialization result of the LBP graph decoder is output. The shared feature encoder, classification head, reflectance map decoder, and LBP map decoder are connected to form a shared encoder-multi-decoder head architecture. A callable interface for the shared encoder-multi-decoder head architecture is established according to a fixed order of classification head initialization results, reflectance map decoder initialization results, and LBP map decoder initialization results, and the network topology and parameter initialization results of the shared encoder-multi-decoder head architecture are output.
[0021] In this embodiment, the generation of the multi-task prediction result set specifically includes: Standardized near-infrared face images are read from training sample units and fed into the shared feature encoder in the shared encoder-multi-decoder head architecture. Convolution stacking, activation operations and scale compression are performed sequentially through multi-level encoding paths to form a multi-scale shared feature set. The multi-scale shared feature set is input into the classification head. First, global average pooling is performed on the fourth layer shared feature map to obtain the intermediate classification representation. Then, linear mapping is performed on the intermediate classification representation to obtain the classification output. The multi-scale shared feature set is input into the reflectance map decoder. Upsampling, feature concatenation and convolution fusion are performed on the multi-scale shared feature set step by step in a top-down path to obtain intermediate features for reflectance map reconstruction. Finally, terminal convolution is performed on the intermediate features for reflectance map reconstruction to generate the reflectance map reconstruction output. The multi-scale shared feature set is input into the LBP graph decoder. Upsampling, feature concatenation and convolution fusion are performed on the multi-scale shared feature set step by step in a top-down path to obtain the intermediate features of LBP graph reconstruction. Finally, terminal convolution is performed on the intermediate features of LBP graph reconstruction to generate the LBP graph reconstruction output. Spatial resolution and tensor channel alignment are performed on the classification output, reflectance map reconstruction output, and LBP map reconstruction output. Based on the output tensor order and spatial resolution alignment rules specified during network setup for the shared encoder-multiple decoder head architecture, the classification output is adjusted to match the classification labels in the training sample units. The reflectance map reconstruction output and LBP map reconstruction output are respectively adjusted to match the reflectance map and LBP map in the training sample units. Figure 1 The spatial dimensions and channel arrangement are aligned to form the aligned classification output, the aligned reflectance map reconstruction output, and the aligned LBP map reconstruction output; The aligned classification output, aligned reflectance map reconstruction output, and aligned LBP map reconstruction output are written into the multi-task prediction result set in a fixed order. The correspondence between these outputs and the training sample unit numbers is established and cached. The multi-task prediction result set is then output.
[0022] In this embodiment, the generation of the converged multi-task model specifically includes: Read the aligned classification output, aligned reflectance map reconstruction output, and aligned LBP map reconstruction output from the multi-task prediction result set, and read the corresponding classification label, reflectance map, and LBP map from the training sample unit, and establish corresponding supervision pairs of classification label, reflectance map, and LBP map according to the sample number; The cross-entropy loss of the main classification task is calculated based on the supervision of the classification label. The loss is accumulated item by item along the category dimension. The indicator value of the classification label in that category is multiplied by the negative of the logarithm of the probability of the classification output in that category. The summation is then applied to all categories to obtain the classification loss. Based on the reflectance map supervision, the pixel-level mean square error loss of the reflectance map reconstruction task is calculated. The total number of pixels is used as the normalization factor. The difference between the true value of each pixel position in the reflectance map and the predicted value of the aligned reflectance map reconstruction output at the same position is squared. The average of the squared differences of all pixel positions is then used to obtain the reflectance map reconstruction loss. The pixel-level mean squared error loss of the LBP image reconstruction task is calculated based on the LBP image supervision. The total number of pixels is used as the normalization factor. The difference between the true value at each position of the LBP image and the predicted value at the same position of the aligned LBP image reconstruction output is squared and averaged to obtain the LBP image reconstruction loss. The classification loss, reflectance map reconstruction loss, and LBP map reconstruction loss are weighted and summed together with preset weights to obtain the joint optimization objective. Backpropagation is performed on the shared encoder-multi-decoder head architecture based on the joint optimization objective. The gradient of the joint optimization objective with respect to the parameters of the shared feature encoder, the classification head, the reflectance map decoder, and the LBP map decoder is calculated. The above parameters are then updated synchronously using a stochastic gradient descent optimizer with momentum. When the convergence criterion, measured by the joint optimization objective, meets a preset threshold, the converged multi-task model is output.
[0023] In this embodiment, the generation of the reasoning model specifically includes: Load the converged multi-task model, read the parameters and network information of the shared feature encoder, classifier head, reflectance map decoder and LBP map decoder, complete the mapping and verification of weights and layer names according to the indexing rules of training sample units, and output a tailorable model description. Structural pruning is performed in the pruning model description, preserving the computational paths of the shared feature encoder and the classification head, removing all levels and parameters of the reflectance map decoder and LBP map decoder, and retaining only the main path structure with the input being a standardized near-infrared face image and the output being a classification, thus outputting a preliminary pruning model; The initial pruning model is subjected to structural optimization and parameter transfer. The parameters corresponding to the shared feature encoder and classifier head in the converged multi-task model are copied to the initial pruning model. The convolutional layer and the batch normalization layer are fused. The identity branch is folded with constants and redundant nodes are removed. The structurally optimized model is output. Perform inference equivalence verification on the structural optimization model, select the sample set of the verification subset to generate classifications using the converged multi-task model and the structural optimization model respectively, compare the class ranking and probability differences of the two on the same sample, and determine equivalence if it does not exceed the preset deviation threshold, and output the inference equivalence model. Using the inference equivalent model as the inference model, the input size, data type and output format are fixed, the input preprocessing requirements and output postprocessing order are recorded, and the inference model and runtime configuration are generated.
[0024] In this embodiment, the generation of the liveness detection result specifically includes: Receive the near-infrared face image to be tested, and perform size standardization, grayscale intensity normalization and face region alignment according to the runtime configuration to obtain a standardized near-infrared face image that meets the inference requirements; The standardized near-infrared face image is input into the inference model, and forward propagation is performed along a single path of the shared feature encoder and the classification head to output the classification result. The classification results are processed probabilistically, the probability distribution corresponding to each category is calculated, and the category with the highest probability is selected as the predicted category. The liveness detection result is then output. The liveness detection result is bound to the corresponding input image identifier to form an inference output record, which is then returned to the application interface.
[0025] A machine learning-based face liveness detection system includes: The near-infrared image acquisition and annotation module is used to acquire and annotate face images and imaging metadata, and generate corresponding reflectance maps and LBP maps offline. The shared encoder building block is used to build a shared feature encoder with the MiniFASNetV2 network as the backbone and build a multi-task decoder head on it. The multi-task prediction module is used to input training sample units into the shared encoder-multi-decoder head architecture and output classification results, reflectance map reconstruction results, and LBP map reconstruction results. The joint training optimization module is used to perform joint loss weighted summation on the multi-task prediction result set based on multi-task joint supervision and update the parameters of the shared feature encoder. The model pruning and inference generation module is used to perform structural pruning on the converged multi-task model to generate a lightweight inference model. The liveness detection module is used to input the near-infrared face image to be tested into the inference model and output the liveness detection result.
[0026] Example 1: To verify the feasibility of this invention in practice, it was applied to the self-service counter identity verification system of a large financial institution. This institution has deployed near-infrared facial recognition terminals in multiple locations across the country for bank card activation, account inquiries, and high-value transaction authentication. However, due to the complex lighting conditions in the business hall, frequent screen reflections, and the ability of some attackers to deceive users using high-fidelity printed photos, high-definition screen photographs, and 3D facial masks, traditional visible light liveness detection algorithms exhibit significant false positives under strong reflections, low light conditions, or different attack media, resulting in persistent security risks. To address this pain point, a machine learning-based facial liveness detection method and system were introduced into this application scenario. Through near-infrared imaging and a multi-task learning mechanism, a liveness detection model with greater physical consistency and generalization ability was constructed.
[0027] During system operation, a near-infrared camera component installed inside the self-service terminal acquires customer facial images using a 940nm active light source. The system automatically generates corresponding reflectance maps and LBP texture maps, forming training sample units. Subsequently, a shared feature encoder with MiniFASNetV2 as the backbone network receives the standardized near-infrared facial image input and simultaneously performs classification output, reflectance reconstruction, and texture reconstruction tasks in the multi-task decoder head. A multi-task joint supervision mechanism is used to weight and optimize the loss, enabling the network to consider both global brightness features and local texture differences when distinguishing between real skin and fake materials, thus achieving robust recognition against different attack methods. After training, the system performs structural pruning on the multi-task model, retaining the shared feature encoder and classification head to form an inference model, allowing it to run in real time even on low-computing-power devices.
[0028] In the counter identity verification process, the customer's face is captured via near-infrared imaging and directly input into the inference model. The system outputs classification results and generates a liveness detection record. This solution maintains high stability under different illumination levels, angles, and various attack conditions, effectively suppressing screen reflection interference and material artifacts. Continuous operation and observation have shown that the system maintains stable response even during peak business hours, significantly reducing false positives and spoofing rates, and significantly improving the security, reliability, and ease of use of the identity verification process.
[0029] Table 1. Performance comparison between machine learning-based face liveness detection methods and traditional methods.
[0030] As shown in Table 1, the method of this invention significantly outperforms existing technologies on the same near-infrared face liveness detection dataset. In terms of average accuracy, this invention achieves 98.9%, a 4.6 percentage point improvement compared to the traditional ResNet18 end-to-end classification method, and approximately a 2.7 percentage point improvement compared to the convolutional model using Retinex illumination enhancement. This demonstrates that the multi-task learning structure of this invention can simultaneously capture reflectivity and local texture, thereby forming a more stable feature representation.
[0031] In terms of the false positive rate, the method of this invention has a false positive rate of only 1.5%, far lower than the 5.7% of traditional CNN methods. This indicates that the model can more accurately distinguish the spectral response of real skin from the material reflectance differences of fake media when facing high-fidelity attacks. This advantage mainly stems from the fact that reflectance maps and LBP texture maps serve as auxiliary supervision signals, imposing physical consistency constraints on the feature extraction process.
[0032] The generalization degradation rate was reduced from approximately 20% in traditional models to 8.4%, demonstrating the model's stability across different devices and lighting conditions. The method of this invention preserves the joint feature space across tasks in the shared feature encoder, enabling the model to maintain consistent feature distribution even when encountering unseen sample domains, thereby significantly reducing cross-domain degradation.
[0033] Although the method of this invention adds a multi-task structure, after pruning and optimization, its inference latency is only 19.2 milliseconds, which is similar to the MobileNetV3 lightweight method. This shows that the inference model after structural optimization still has real-time response capability in low computing power environment, balancing performance and efficiency.
[0034] Comprehensive analysis shows that the present invention, through multi-task joint supervision of a shared encoder-multi-decoder head architecture, not only significantly improves the accuracy and robustness of liveness detection, but also maintains the lightweight and high efficiency of the model, demonstrating outstanding practical value in scenarios such as secure financial authentication, access control security, and high-fidelity identity verification.
[0035] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A face liveness detection method based on machine learning, characterized in that, Includes the following steps: Near-infrared face images are acquired and labeled, and corresponding auxiliary supervision images are generated offline to form training sample units; A shared feature encoder with the MiniFASNetV2 network as the backbone is constructed, and a multi-task decoder head is built on it to form a shared encoder-multi-decoder head architecture. The training sample units are input into the shared encoder-multi-decoder head architecture to obtain classification output, reflectance map reconstruction output and LBP map reconstruction output, and generate a set of multi-task prediction results. Based on multi-task joint supervision, the joint loss weighted summation is performed on the multi-task prediction result set, and the parameters of the shared feature encoder are updated using a stochastic gradient descent optimizer to obtain the converged multi-task model. The converged multi-task model is pruned to obtain the inference model. The near-infrared face image to be tested is input into the inference model to obtain classification and generate a liveness detection result after post-processing.
2. The face liveness detection method based on machine learning according to claim 1, characterized in that, The generation of the training sample units specifically includes: Under near-infrared imaging conditions, facial images and their imaging metadata are acquired. The acquired near-infrared facial images are labeled as real or fake, and labels are established according to different presentation attack subclasses. A set of near-infrared facial images with real and fake labels and presentation attack subclass labels is output. The near-infrared imaging conditions are imaging using an active light source of 850nm or 940nm, and the attack subclasses include printed photos, screen recordings, and 3D masks. The near-infrared face image set is processed for format unification and spatial alignment, including size standardization, grayscale intensity normalization and face region alignment, to generate a standardized near-infrared face image set; Based on the Retinex concept, the illumination component is obtained by convolving the face image with the Gaussian blur kernel. Then, the gray value of the standardized near-infrared face image corresponding to the face image is divided by the illumination component and a stabilization term is added to obtain a candidate set of reflectance maps. LBP images are extracted from each image in the standardized near-infrared face image set. Local binary patterns are calculated using a unified neighborhood and radius configuration. The grayscale difference between the center pixel and the neighboring pixels is compared and the result is encoded into a binary pattern to obtain local texture features, thus obtaining a candidate set of LBP images. Size alignment and naming alignment are performed on the candidate set of reflectance map and candidate set of LBP map. An index mapping table is built for each standardized near-infrared face image and its corresponding reflectance map and LBP map according to the correspondence. The training index list containing file path, true and false labels and attack subclass labels is output. The samples are divided according to the training index list. A hierarchical partitioning method is used to generate training subsets and validation subsets while maintaining the ratio of true to false and the distribution of attack subclasses. The hierarchical partitioning results are output. Standardized near-infrared face images, their corresponding reflectance maps and LBP maps, as well as true and false labels and presentation attack subclass labels from the training index list, are encapsulated into training sample units according to sample number.
3. The face liveness detection method based on machine learning according to claim 1, characterized in that, The generation of the shared encoder-multi-decoder head architecture specifically includes: A shared feature encoder is constructed using the MiniFASNetV2 network as the backbone. The MiniFASNetV2 network consists of an input layer, a convolutional normalized activation layer, four groups of depthwise separable convolutional modules and residual convolutional modules. Downsampling is achieved between each group through convolution with a stride of 2, and the output size is halved layer by layer. A classification head is built in the multi-task decoder head, which is composed of a global average pooling layer, a fully connected layer and a softmax layer connected in sequence. It is used to map the high-dimensional features output by the shared feature encoder to the probability output of true and false binary classification, and output the classification head initialization result. A reflectance map decoder is established in the multi-task decoder head, which consists of a multi-level upsampling layer, a convolutional fusion layer and a terminal convolutional layer. The reflectance distribution of the near-infrared face image is reconstructed by step-by-step upsampling and cross-layer stitching, and the initialization result of the reflectance map decoder is output. An LBP graph decoder is built in the multi-task decoder head, consisting of multi-level upsampling layers, convolutional fusion layers and terminal convolutional layers. The LBP texture features are reconstructed by feature upsampling at each level and cross-layer stitching, and the initialization result of the LBP graph decoder is output. The shared feature encoder, classification head, reflectance map decoder, and LBP map decoder are connected to form a shared encoder-multi-decoder head architecture. A callable interface for the shared encoder-multi-decoder head architecture is established according to a fixed order of classification head initialization results, reflectance map decoder initialization results, and LBP map decoder initialization results, and the network topology and parameter initialization results of the shared encoder-multi-decoder head architecture are output.
4. The face liveness detection method based on machine learning according to claim 1, characterized in that, The generation of the multi-task prediction result set specifically includes: Standardized near-infrared face images are read from training sample units and fed into the shared feature encoder in the shared encoder-multi-decoder head architecture. Convolution stacking, activation operations and scale compression are performed sequentially through multi-level encoding paths to form a multi-scale shared feature set. The multi-scale shared feature set is input into the classification head. First, global average pooling is performed on the fourth layer shared feature map to obtain the intermediate classification representation. Then, linear mapping is performed on the intermediate classification representation to obtain the classification output. The multi-scale shared feature set is input into the reflectance map decoder. Upsampling, feature concatenation and convolution fusion are performed on the multi-scale shared feature set step by step in a top-down path to obtain intermediate features for reflectance map reconstruction. Finally, terminal convolution is performed on the intermediate features for reflectance map reconstruction to generate the reflectance map reconstruction output. The multi-scale shared feature set is input into the LBP graph decoder. Upsampling, feature concatenation and convolution fusion are performed on the multi-scale shared feature set step by step in a top-down path to obtain the intermediate features of LBP graph reconstruction. Finally, terminal convolution is performed on the intermediate features of LBP graph reconstruction to generate the LBP graph reconstruction output. Spatial resolution and tensor channel alignment are performed on the classification output, reflectance map reconstruction output, and LBP map reconstruction output. According to the output tensor order and spatial resolution alignment rules specified by the shared encoder-multi-decoder head architecture during networking, the classification output is adjusted to be consistent with the classification label in the training sample unit. The reflectance map reconstruction output and LBP map reconstruction output are adjusted to be consistent with the spatial size and channel arrangement of the reflectance map and LBP map in the training sample unit, respectively, to form the aligned classification output, aligned reflectance map reconstruction output, and aligned LBP map reconstruction output. The aligned classification output, aligned reflectance map reconstruction output, and aligned LBP map reconstruction output are written into the multi-task prediction result set in a fixed order. The correspondence between these outputs and the training sample unit numbers is established and cached. The multi-task prediction result set is then output.
5. The face liveness detection method based on machine learning according to claim 1, characterized in that, The generation of the converged multi-task model specifically includes: Read the aligned classification output, aligned reflectance map reconstruction output, and aligned LBP map reconstruction output from the multi-task prediction result set, and read the corresponding classification label, reflectance map, and LBP map from the training sample unit, and establish corresponding supervision pairs of classification label, reflectance map, and LBP map according to the sample number; The cross-entropy loss of the main classification task is calculated based on the supervision of the classification label. The loss is accumulated item by item along the category dimension. The indicator value of the classification label in that category is multiplied by the negative of the logarithm of the probability of the classification output in that category. The summation is then applied to all categories to obtain the classification loss. Based on the reflectance map supervision, the pixel-level mean square error loss of the reflectance map reconstruction task is calculated. The total number of pixels is used as the normalization factor. The difference between the true value of each pixel position in the reflectance map and the predicted value of the aligned reflectance map reconstruction output at the same position is squared. The average of the squared differences of all pixel positions is then used to obtain the reflectance map reconstruction loss. The pixel-level mean squared error loss of the LBP image reconstruction task is calculated based on the LBP image supervision. The total number of pixels is used as the normalization factor. The difference between the true value at each position of the LBP image and the predicted value at the same position of the aligned LBP image reconstruction output is squared and averaged to obtain the LBP image reconstruction loss. The classification loss, reflectance map reconstruction loss, and LBP map reconstruction loss are weighted and summed together with preset weights to obtain the joint optimization objective. Backpropagation is performed on the shared encoder-multi-decoder head architecture based on the joint optimization objective. The gradient of the joint optimization objective with respect to the parameters of the shared feature encoder, the classification head, the reflectance map decoder, and the LBP map decoder is calculated. The above parameters are then updated synchronously using a stochastic gradient descent optimizer with momentum. When the convergence criterion, measured by the joint optimization objective, meets a preset threshold, the converged multi-task model is output.
6. The face liveness detection method based on machine learning according to claim 1, characterized in that, The generation of the inference model specifically includes: Load the converged multi-task model, read the parameters and network information of the shared feature encoder, classifier head, reflectance map decoder and LBP map decoder, complete the mapping and verification of weights and layer names according to the indexing rules of training sample units, and output a tailorable model description. Structural pruning is performed in the pruning model description, preserving the computational paths of the shared feature encoder and the classification head, removing all levels and parameters of the reflectance map decoder and LBP map decoder, and retaining only the main path structure with the input being a standardized near-infrared face image and the output being a classification, thus outputting a preliminary pruning model; The initial pruning model is subjected to structural optimization and parameter transfer. The parameters corresponding to the shared feature encoder and classifier head in the converged multi-task model are copied to the initial pruning model. The convolutional layer and the batch normalization layer are fused. The identity branch is folded with constants and redundant nodes are removed. The structurally optimized model is output. Perform inference equivalence verification on the structural optimization model, select the sample set of the verification subset to generate classifications using the converged multi-task model and the structural optimization model respectively, compare the class ranking and probability differences of the two on the same sample, and determine equivalence if it does not exceed the preset deviation threshold, and output the inference equivalence model. Using the inference equivalent model as the inference model, the input size, data type and output format are fixed, the input preprocessing requirements and output postprocessing order are recorded, and the inference model and runtime configuration are generated.
7. The face liveness detection method based on machine learning according to claim 1, characterized in that, The generation of the liveness detection result specifically includes: Receive the near-infrared face image to be tested, and perform size standardization, grayscale intensity normalization and face region alignment according to the runtime configuration to obtain a standardized near-infrared face image that meets the inference requirements; The standardized near-infrared face image is input into the inference model, and forward propagation is performed along a single path of the shared feature encoder and the classification head to output the classification result. The classification results are processed probabilistically, the probability distribution corresponding to each category is calculated, and the category with the highest probability is selected as the predicted category. The liveness detection result is then output. The liveness detection result is bound to the corresponding input image identifier to form an inference output record, which is then returned to the application interface.
8. A machine learning-based face liveness detection system, comprising executing the machine learning-based face liveness detection method according to any one of claims 1 to 7, characterized in that, include: The near-infrared image acquisition and annotation module is used to acquire and annotate face images and imaging metadata, and generate corresponding reflectance maps and LBP maps offline. The shared encoder building block is used to build a shared feature encoder with the MiniFASNetV2 network as the backbone and build a multi-task decoder head on it. The multi-task prediction module is used to input training sample units into the shared encoder-multi-decoder head architecture and output classification results, reflectance map reconstruction results, and LBP map reconstruction results. The joint training optimization module is used to perform joint loss weighted summation on the multi-task prediction result set based on multi-task joint supervision and update the parameters of the shared feature encoder. The model pruning and inference generation module is used to perform structural pruning on the converged multi-task model to generate a lightweight inference model. The liveness detection module is used to input the near-infrared face image to be tested into the inference model and output the liveness detection result.