Living body classifier establishment method, face living body detection method and device
By preprocessing the image and extracting the camera's invariant and enhanced features, combined with the loss function, the live classifier is optimized, and the problem of insufficient generalization ability of convolutional neural networks in facial live detection is solved, and more accurate live detection and safety improvement.
Patent Information
- Application Number
- CN202011450129.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-11
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2040-12-11
AI Technical Summary
The existing convolutional neural network's face live detection method will leave traces of processing when obtaining visual information, resulting in lack of generalization ability and being unable to accurately distinguish between real and fraudulent faces, and the usage scenarios are limited.
通过对采集的图像进行预处理,提取高频信息和增强图像,利用伪孪生神经网络从高频信息中确定相机不变的活体特征,并从增强图像中提取增强的活体特征,结合二分类焦点 Loss function and pixel-by-pixel point multi-class focus loss function optimize the living classifier, eliminate camera information, and improve the generalization ability of living detection.
It has achieved the elimination of differences in vivo discrimination characteristics caused by different collection equipment, improved the generalization ability of vivo detection, prevented various attacks, and ensured the safety of vivo detection in human faces.
Smart Images

Figure CN114694196B_ABST
Abstract
Description
Technical Field
[0001] This article relates to the field of face recognition, and in particular to a method for establishing a liveness classifier, a method and a device for detecting liveness of a face. Background Art
[0002] Facial recognition plays an increasingly important role in financial technology. However, facial recognition algorithms are very vulnerable to fake face attacks, and fake face information is very easy to obtain.
[0003] With the development and maturity of face recognition technology, face verification has gradually become a common verification method for security systems, and liveness detection for fraudulent faces has also become an important part of security systems. Existing face liveness detection solutions can be divided into two categories, including solutions that perform liveness detection by extracting manual liveness features and solutions that perform liveness detection through neural networks. Manual liveness features such as Local Binary Patterns (LBP) features, Speeded Up Robust Features (SURF) features, Histogram of Gradients (HoG) features, etc. often have lower accuracy in liveness discrimination than solutions based on neural network feature extraction. Among the existing liveness detection solutions based on convolutional neural networks, some treat liveness detection as a binary classification problem, and use pre-trained ResNet and Inception networks for fine-tuning through training of data sets to obtain more discriminative liveness features. At the same time, some work further improves the accuracy of network liveness detection by extracting auxiliary information such as depth information and rPPG information. However, although convolutional neural networks can achieve high accuracy within a data set, when tested across data sets (data collected by different cameras), the accuracy often drops significantly. This is because when visual information (images or videos) is acquired, unique traces of hardware and software processing are left in the visual information. These unique traces are usually located in the high-frequency domain (for example, sensor overlap noise). However, important clues of face fraud, such as moiré in video replay attacks or the texture of masks made of artificial materials (paper, facial masks), also belong to the high-frequency domain. Therefore, while the convolutional neural network model used for face recognition extracts high-frequency features, it is inevitable to introduce camera information, so that the proposed live features are often based on specific cameras, thereby reducing the generalization ability of the convolutional neural network model under different camera configurations and limiting the use scenarios of the convolutional neural network model. In addition, the existing convolutional neural network model for face recognition is not easy to distinguish between real faces and fraudulent faces, and has the defect of poor security. Summary of the invention
[0004] This article aims to solve the problem that the convolutional neural network face liveness detection method in the existing technology leaves processing traces in the visual information when acquiring visual information. These traces make the existing convolutional neural network liveness detection method lack generalization ability, have limited usage scenarios, and cannot accurately distinguish between real faces and fraudulent faces.
[0005] In order to solve the above technical problems, the first aspect of this paper provides a method for establishing a living body classifier, comprising:
[0006] Preprocess the collected images to obtain high-frequency information and enhanced images;
[0007] Using the first network, determine the camera-invariant living feature M from the high-frequency information spf ; Using the second network to determine the enhanced living feature M from the enhanced image aug ;
[0008] The camera-invariant living feature M spf And the enhanced living feature M aug Input them into the liveness classifier respectively to get the liveness prediction probability p of the first face spf And the second face live prediction probability p aug ;
[0009] According to the face label, the first face live prediction result p spf And the second face live prediction result p aug , construct a binary classification focal loss function and
[0010] Using the binary focal loss function and Optimizing parameters in the living body classifier, the first network, and the second network.
[0011] In one embodiment of the present invention, the collected image is preprocessed to obtain high-frequency information and an enhanced image, including:
[0012] Cutting out a facial region image from the collected image;
[0013] The R, G, and B channels of the facial region image are processed using N directional differential filters to obtain 3N feature maps M diff ;
[0014] According to the feature map M diff , determine the high-frequency information M hf and image enhancement component I aug ;
[0015] Combined with the image enhancement component Iaug and the collected image to obtain an enhanced image.
[0016] In one embodiment of this invention, according to the characteristic graph M diff , determine the high-frequency information M hf and image enhancement component I aug include:
[0017] Using M*M channel convolutional neural network Conv hf For each of the 3N feature maps M diff Learn and get high-frequency information M hf ;
[0018] Using three-channel convolutional neural network Conv aug For the 3N feature map M diff The feature maps of the R, G, and B channels are learned to obtain the image enhancement component I aug .
[0019] In one embodiment of the present invention, the first network includes a first sub-network and a second sub-network with the same structure and different parameters, and the first network is used to determine the camera-invariant living feature M from the high-frequency information. spf ,include:
[0020] The high-frequency information is input into the first sub-network of the first network to obtain the camera information M cam ;
[0021] The high-frequency information is input into the second sub-network of the first network to obtain the fusion information M of the camera and the living body. mix ;
[0022] Using the fusion information M of the camera and the living body mix Subtract the camera information M cam , and obtain the camera-invariant living feature M spf .
[0023] In one embodiment of the present invention, the method for establishing a living body classifier further includes:
[0024] The camera information M cam Input into the camera classification network to obtain the camera classification prediction probability P cam ; According to the camera label and the camera classification prediction probability P cam , construct a pixel-by-pixel multi-class focal loss function and / or
[0025] The fusion information M of the camera and the living body mix Input into the camera classification network to obtain the camera classification prediction probability P mix ; According to the camera classification prediction probability Pmix , construct a pixel-by-pixel multi-class focal loss function and / or
[0026] The camera-invariant living feature M spf Input into the camera classification network to obtain the camera classification prediction probability P spf , according to the camera classification prediction probability P spf , construct the camera loss function and / or
[0027] The fusion information M of the camera and the living body mix Input into the liveness classifier to obtain the liveness prediction probability p of the third face mix ; Based on the face label and the third face live prediction probability p mix , construct a binary classification focal loss function
[0028] According to the binary focal loss function and Combining pixel-by-pixel multi-class focal loss functions and / or pixel-wise multi-class focal loss function and / or go to camera loss function and / or binary focal loss function Construct the target loss function;
[0029] Using the binary focal loss function and Optimizing the parameters of the living body classifier, the first network and the second network further includes: optimizing the parameters of the living body classifier, the first network and the second network using the target loss function.
[0030] In one embodiment of this invention, if the target loss function is a binary focal loss function and Combining pixel-by-pixel multi-class focal loss functions Pixel-wise multi-class focal loss function Camera loss function Binary classification focal loss function The objective loss function is expressed by the following formula:
[0031]
[0032] Among them, λ1, λ2 and λ3 are weighting coefficients.
[0033] In one embodiment of the present invention, the second network is used to determine the enhanced living feature M from the enhanced image. aug ,include:
[0034] The enhanced image is input into the second network to obtain the enhanced living feature M aug .
[0035] In one embodiment of the present invention, the camera-invariant living feature M spf And the enhanced living feature M aug Before inputting into the living body classifier, it also includes:
[0036] The camera-invariant living feature M spf And the enhanced living feature M aug Perform average pooling.
[0037] The second aspect of the present invention provides a method for detecting liveness of a face, comprising:
[0038] Preprocess the real-time collected images to obtain high-frequency information and enhanced images;
[0039] Using the first network, determine the camera-invariant living feature M from the high-frequency information spf ; Using the second network to determine the enhanced living feature M from the enhanced image aug ;
[0040] The camera-invariant living feature M spf And the enhanced living feature M aug Input them into the liveness classifier respectively to obtain the liveness prediction result of the first face and the liveness prediction result of the second face;
[0041] Combining the first face liveness prediction result and the second face liveness prediction result to obtain a face liveness prediction result;
[0042] The first network, the second network and the living body classifier are trained by the living body classifier establishment method described in any of the aforementioned embodiments.
[0043] A third aspect of the present invention provides a face liveness detection device, comprising:
[0044] A storage module, used for storing a living body classifier established by using the living body classifier establishment method described in any of the above embodiments;
[0045] A preprocessing module is used to preprocess the images collected in real time to obtain high-frequency information and enhanced images;
[0046] An extraction module is used to determine the camera-invariant living feature M from the high-frequency information using the first network. spf ; Using the second network to determine the enhanced living feature M from the enhanced image aug ;
[0047] A prediction module is used to transform the camera-invariant living feature M spf And the enhanced living feature M aug respectively inputting them into the liveness classifier to obtain a first face liveness prediction result and a second face liveness prediction result;
[0048] A combination module is used to combine the first face liveness prediction result and the second face liveness prediction result to obtain a face liveness prediction result.
[0049] The fourth aspect of this document provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for establishing a living body classifier described in any of the aforementioned embodiments is implemented.
[0050] The fifth aspect of this article provides a computer-readable storage medium, which stores an executable computer program, and when the computer program is executed by the processor, it implements the method for establishing a liveness classifier described in any of the above embodiments. The liveness classifier establishment method, face liveness detection method and device proposed in this article obtain high-frequency information and enhanced images by preprocessing the collected images; extracting camera-invariant liveness features from the high-frequency information, and extracting enhanced liveness features from the enhanced images; using the liveness classifier to identify the camera-invariant liveness features and enhanced liveness features, respectively, to obtain the first and second face liveness prediction probabilities; combining the first and second face liveness prediction probabilities to obtain the final face liveness prediction probability, which can eliminate the differences in liveness discrimination features caused by different acquisition devices in face liveness detection, improve the generalization ability of liveness detection, and in addition, can prevent various attacks (photo attacks, replay attacks, mask attacks, etc.) to ensure the security of face liveness detection.
[0051] In order to make the above and other purposes, features and advantages of this article more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the technical solutions in the embodiments of this article or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of this article. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0053] Figure 1 A flow chart of a method for establishing a living body classifier according to an embodiment of this invention is shown;
[0054] Figure 2A flow chart showing the preprocessing process of the embodiment of this article is shown;
[0055] Figure 3 The network structure diagram required for establishing the living body classifier in the embodiment of this article is shown;
[0056] Figure 4 A schematic diagram of an eight-directional differential filter according to an embodiment of the present invention is shown;
[0057] Figure 5 A flow chart of a method for detecting live faces in an embodiment of this invention is shown;
[0058] Figure 6 The structure diagram of the face liveness detection device in the embodiment of this article is shown;
[0059] Figure 7 A structural diagram of a computer device according to an embodiment of the present invention is shown.
[0060] Description of the accompanying symbols:
[0061] 610. Storage module;
[0062] 620, preprocessing module;
[0063] 630, extraction module;
[0064] 640, prediction module;
[0065] 650, combined module;
[0066] 702. Computer equipment;
[0067] 704, processor;
[0068] 706. Memory;
[0069] 708, driving mechanism;
[0070] 710, input / output module;
[0071] 712. Input devices;
[0072] 714. Output devices;
[0073] 716. Presentation equipment;
[0074] 718. Graphical user interface;
[0075] 720, network interface;
[0076] 722. Communication link;
[0077] 724. Communication bus. DETAILED DESCRIPTION
[0078] The following will be combined with the drawings in the embodiments of this article to clearly and completely describe the technical solutions in the embodiments of this article. Obviously, the described embodiments are only part of the embodiments of this article, not all of the embodiments. Based on the embodiments of this article, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this article.
[0079] In one embodiment of this invention, Figure 1 As shown, Figure 1 The flowchart of the method for establishing a liveness classifier in the embodiment of this article is shown. The liveness classifier established in this embodiment can eliminate the defects of the existing convolutional neural network for liveness detection, such as lack of generalization ability, limited usage scenarios, and inability to accurately distinguish between real faces and fraudulent faces. Specifically, the method for establishing a liveness classifier includes:
[0080] Step 110, preprocessing the collected image to obtain high-frequency information and enhanced image;
[0081] Step 120, using the first network to determine the camera-invariant living feature M from the high-frequency information spf ; Using the second network to determine the enhanced living feature M from the enhanced image aug ;
[0082] Step 130: convert the camera-invariant living feature M spf And the enhanced living feature M aug Input them into the liveness classifier respectively to get the liveness prediction probability p of the first face spf And the second face live prediction probability p aug ;
[0083] Step 140: based on the face label and the first face liveness prediction result p spf And the second face live prediction result p aug , construct a binary classification focal loss function and
[0084] Step 150, using the binary classification focal loss function and The parameters of the living body classifier, the first network and the second network are trained.
[0085] Specifically, the images collected in step 110 all contain human faces and are pre-collected images, including images collected by various types of collection devices, such as collection devices of different manufacturers and models. High-frequency information contains a large number of clues about camera and living body related features, and enhanced images contain low-frequency / high-frequency signal information, which can reflect face fraud clues such as brightness and reflection.
[0086] The first network and the second network in step 120 are neural networks. When implemented, the structures of the first network and the second network can be set according to the requirements or the existing neural network structures can be selected. In one embodiment, in order to align the living features captured by different acquisition devices to obtain the camera-invariant living feature M spf The first network uses a pseudo twin neural network, which includes two identical structures, the first sub-network and the second sub-network, such as the ResNet-18 network. Different from the twin neural network, the weights of the first sub-network and the second sub-network of the pseudo twin neural network are different, because the first sub-network and the second sub-network share the same camera classification network, so the first sub-network and the second sub-network will extract the same camera information. The goal of the first sub-network is to extract the camera information M cam , the goal of the second sub-network is to extract the fusion information M of the camera and the living body mix Specifically, the high-frequency information is input into the first sub-network of the first network to obtain the camera information M cam , input the high-frequency information into the second sub-network of the first network to obtain the fusion information M of the camera and the living body mix , using the fusion information M of the camera and the living body mix Subtract the camera information M cam , and obtain the camera-invariant living feature M spf .
[0087] The second network recombines the low-frequency / high-frequency signal learning of the original image to enhance the extraction of face fraud clues, thereby performing more discriminative feature learning.
[0088] In the above step 130, the camera-invariant living feature M spf And the enhanced living feature M aug Input different living body classifiers F respectively spf and F aug , live classifier F spf and F aug The network structure is the same but the specific parameters are different, for example, a neural network composed of two fully connected layers and a softmax function. This article does not limit this specific structure.
[0089] In the above step 140, according to the first face liveness prediction result p spf And face labels, constructing the binary classification focal loss function as follows:
[0090]
[0091] Among them, y spf is the face label, α1, α2, γ are constant parameters used to control the loss balance between samples.
[0092] According to the second face liveness prediction result p aug And face labels, constructing the binary classification focal loss function as follows:
[0093]
[0094] Among them, y aug is the face label, α1, α2, γ are constant parameters used to control the loss balance between samples.
[0095] In specific implementation, spf With y aug The values are the same, which are all real face labels and can be identified manually. spf and aug If y is equal to 0, it means the sample is a fraudulent face. spf and aug A value equal to 1 indicates that the sample is a living face.
[0096] In the above step 150, the parameters in the living body classifier, the first network and the second network may be trained using an Adam optimizer, or gradient descent methods such as SGD, RMSprop and Adagrad may be used for training. The specific training method is not limited herein.
[0097] This embodiment can eliminate the differences in liveness discrimination features caused by different acquisition devices in face liveness detection, improve the generalization ability of liveness detection, and also prevent various attacks (photo attacks, replay attacks, mask attacks, etc.) to ensure the security of face liveness detection.
[0098] In one embodiment of the present invention, the above step 110 can use a fixed convolution kernel to extract high-frequency features. The fixed convolution kernel is, for example, a LoG operator convolution, a Sobel operator convolution kernel, a DoG operator convolution kernel, etc. In specific implementation, in order to improve the diversity of high-frequency information extraction, such as Figure 2 and Figure 3 As shown, the above step 110 pre-processes the collected image to obtain high-frequency information and an enhanced image, including:
[0099] Step 210, cropping a facial region image from the collected image;
[0100] Step 220, use N directional differential filters to process the R, G, and B channels of the facial region image to obtain 3N feature maps M diff ;
[0101] Step 230, according to the feature map M diff , determine the high-frequency information M hf and image enhancement component Iaug ;
[0102] Step 240, combining the image enhancement component I aug and the collected images to obtain an enhanced image.
[0103] In detail, step 210 can use existing face recognition software to crop the facial area image, and this document does not limit the specific cropping process.
[0104] The differential filter used in step 220 is a 2D convolution with a fixed convolution kernel, such as Figure 4 As shown, an eight-directional differential filter (EDDF) is preferred. The eight-directional differential filter is composed of differential convolution kernels with the center pixel as the center and the eight directions of the adjacent images being fixed. The order of the eight-directional differential filters can be arbitrary. Figure 4 Just one possible sorting option.
[0105] The above step 230 is based on the feature map M diff , determine the high-frequency information M hf and image enhancement component I aug include:
[0106] Using M×M channel convolutional neural network Conv hf For each 3N feature map M diff Learn and get high-frequency information M hf ;
[0107] Using three-channel convolutional neural network Conv aug For 3N feature map M diff The feature maps of the R, G, and B channels are learned to obtain the image enhancement component I aug . In implementation, the image enhancement component I aug The enhanced image is obtained by superimposing it with the collected image.
[0108] In some implementations, consider the channel convolutional neural network Conv hf To improve the timeliness and performance, M×M is set to 5×5. During the specific implementation, M×M can be set to other values according to the requirements, which is not limited in this article.
[0109] In one embodiment of the present invention, in order to improve the recognition accuracy, before the above step 130 is implemented, the camera-invariant living feature M is required to be spf and enhanced live features aug Dimensionality reduction is performed. Specifically, the camera-invariant living feature M spf and enhanced live features aug Average pooling is performed to achieve dimensionality reduction.
[0110] In one embodiment of this invention, considering the camera information M cam It is determined by the camera type. When implementing, the classification loss of the camera type can be used to guide the acquisition of camera information. Figure 3 As shown, the camera information M is obtained. cam Afterwards, the camera classification network Conv of the CNN layer is used cam Camera information M cam Identify and obtain the camera classification prediction probability P cam , where the camera classification network Conv cam There are Φ channels, where Φ is the number of camera categories contained in the sampled image. In order to mine difficult samples (difficult samples refer to samples that are difficult to predict accurately or have low prediction accuracy), the last layer of the CNN layer O cam We design a pixel-by-pixel multi-class focal loss function For camera type identification:
[0111]
[0112]
[0113] Among them, M and N represent O cam The size of the space, is the binary label of the camera type, when O cam When the element at (i, j) belongs to camera type k, otherwise γ is the penalty coefficient for simple samples (simple samples refer to samples that are easy to predict accurately). The probability that the output of (i, j) belongs to camera type k is, O cam The output of the kth channel at the spatial position (i, j) in the image. γ can increase the loss weight of difficult samples relative to simple samples, thereby increasing the classification distance between different categories and improving the judgment ability of the camera classification network.
[0114] In this embodiment, a pixel-by-pixel multi-class focus loss function is designed. The extracted camera information can be supervised pixel by pixel, which increases the diversity of samples (each position in space is a sample). In addition, the camera information should be independent of the scene being photographed. Classifying the same camera type at multiple positions in an image can improve the generalization ability of camera information extraction.
[0115] In a further embodiment of this invention, considering the fusion information M from the camera and the living body mix The living features extracted from the image are mixed with the camera. In order to eliminate the difference in living features caused by the camera, the same camera classification network Conv camExtract the fusion information M of the camera and the living body mix The camera information in the camera classifier is used to obtain the camera classification prediction probability P mix , and in the last layer of the CNN layer O mix Design a pixel-by-pixel multi-class focal loss function
[0116]
[0117] Among them, M and N represent O mix The size of the space, is the binary label of the camera type, when O mix When the element at (i, j) belongs to camera type k, otherwise γ is the penalty coefficient for simple samples, The probability that the output of (i, j) belongs to camera type k is, O mix The output of the kth channel at spatial position (i, j) in .
[0118] In this embodiment, a pixel-by-pixel multi-class focus loss function is designed. Able to maximize the extraction of M mix The camera information in the image can eliminate the differences in living features caused by the camera.
[0119] In one embodiment of this invention, considering the camera-invariant living feature M spf While having a strong ability to detect living things, the camera model should be invariant. To this end, the same camera classification network Conv cam Extract camera-invariant living features M spf The camera information in the camera classifier is used to obtain the camera classification prediction probability P spf , and in the last layer of the CNN layer O spf The camera loss function L is designed above. decam :
[0120]
[0121]
[0122] Among them, M and N represent O spf The size of the space, is the average probability of camera type, γ is the penalty coefficient of simple samples, The probability that (i, j) belongs to camera type k is output.
[0123] This embodiment can maximize the removal of M by removing the camera loss function. spfThe camera information in M makes the predicted probability of each camera type extracted equal to the average value, which indicates that M spf The component used for camera discrimination is lost.
[0124] In specific implementation, in addition to using the camera loss function L decam In addition to removing information, the camera information can also be removed by using the Gradient Reversal Layer (GRL). Specifically, the GRL network will be used to determine M spf The gradient generated by the camera category information is reversed to achieve the purpose of maximizing the removal of camera information.
[0125] In one embodiment of this invention, in order to perform living body identification, Figure 3 As shown, average pooling is used for M mix Perform dimensionality reduction, obtain feature labels, and then pass the living body classifier F mix (composed of two fully connected layers and a softmax function) acts on M mix The feature markers obtained by dimensionality reduction are used to obtain the predicted value p of the probability of the input face being alive. mix , and adopts the binary classification focal loss function Train the network:
[0126]
[0127] where y mix is the true label, y mix =0 means the sample is a fraudulent face, y mix =1 means it is a living face, p mix is the probability that the network predicts that the sample is a living face. α1, α2 and γ are constant parameters used to control the loss balance between difficult samples and easy samples.
[0128] This embodiment uses a binary classification focus loss function The distance between real and spoofed samples can be increased, so that more discriminative spoofing features can be learned.
[0129] In one embodiment of the present invention, the above step 150 uses a binary classification focal loss function and The parameters in training the living body classifier, the first network and the second network are further:
[0130] According to the binary focal loss function and Combining pixel-by-pixel multi-class focal loss functions and / or pixel-wise multi-class focal loss function and / or go to camera loss function and / or binary focal loss function Construct a target loss function; use the target loss function to at least optimize the living body classifier (F spf 、F aug ), parameters in the first network and the second network.
[0131] In one specific embodiment, if the target loss function is a binary focal loss function and Combining pixel-by-pixel multi-class focal loss functions Pixel-wise multi-class focal loss function Camera loss function Binary classification focal loss function The objective loss function is expressed by the following formula:
[0132]
[0133] Among them, λ1, λ2 and λ3 are weighting coefficients.
[0134] Through the above objective function, the living body classifier (F mix 、F spf 、F aug ), the first network, the second network and the camera classification network Conv cam In addition, the M*M channel convolutional neural network Conv can also be optimized hf The parameters in and the three-channel convolutional neural network Conv aug Parameters in .
[0135] In one embodiment of the present invention, a method for detecting live faces is provided. The method can be run on a smart terminal, including a smart phone, a tablet computer, a desktop computer, etc. It can be a separate application, a small program embedded in other programs, etc., or it can be in the form of a web page, etc. Specifically, Figure 5 As shown, the face liveness detection method includes:
[0136] Step 510, preprocessing the real-time collected image to obtain high-frequency information and enhanced image;
[0137] Step 520, using the first network to determine the camera-invariant living feature M from the high-frequency information spf ; Using the second network to determine the enhanced living feature M from the enhanced image aug ;
[0138] Step 530: convert the camera-invariant living feature M spf And the enhanced living feature M augInput them into the liveness classifier respectively to obtain the liveness prediction result of the first face and the liveness prediction result of the second face;
[0139] Step 540, combining the first face liveness prediction result and the second face liveness prediction result to obtain a face liveness prediction result;
[0140] The first network, the second network and the living body classifier are trained by the living body classifier establishment method described in any of the aforementioned embodiments.
[0141] The processing process of the above step 510 is referred to the above embodiment and will not be described again in detail herein.
[0142] In the above step 540, the first face liveness prediction result and the second face liveness prediction result may be combined using the following formula:
[0143]
[0144] Among them, λ4 is the fusion score weight set according to experience, P(F spf ) and P(F aug ) are respectively the prediction probability of the first face being alive and the prediction probability of the second face being alive.
[0145] This embodiment can eliminate the differences in liveness discrimination features caused by different acquisition devices in face liveness detection, improve the generalization ability of liveness detection, and also prevent various attacks (photo attacks, replay attacks, mask attacks, etc.) to ensure the security of face liveness detection.
[0146] In order to more clearly illustrate the technical solution of this invention, a specific embodiment is described in detail below. As shown in Table 1, Table 1 shows Figure 3 Parameters required for the network structure shown. In order to make the training more stable, group normalization is used for the normalization module, and the number of groups is set to 32.
[0147] Table 1
[0148]
[0149] In this embodiment, the high-frequency information preprocessing process and the first network are divided into the first type of network, and the enhanced image preprocessing process and the second network are divided into the second type of network. The residual layer in the first type of network in Table 1 corresponds to the first network, and the residual layer in the second type of network corresponds to the second network.
[0150] It should be noted that Table 1 is only a specific implementation method. When this article is implemented, other network structures can also be designed according to needs.
[0151] Figure 3The training database required for the network structure shown stores videos. During implementation, an average of 30 frames are selected from each video (in other implementations, other numbers of frames can also be selected, such as 35 frames, 40 frames, etc.). The face area is cropped from each frame using a face detection algorithm, and the cropped face image is adjusted to 224×224. During training, live faces and fraudulent faces are sampled at a ratio of 1:1, and data enhancement is used to improve generalization. For data enhancement, the input samples are flipped horizontally and vertically with a probability of 0.5, and randomly rotated within 15 degrees and color jittered.
[0152] The batch size of the training phase is 32, and this embodiment uses the Adam optimizer for optimization. For the first 20k steps, the learning rate is kept at 0.004, and then every 10k steps, the learning rate is reduced by 0.2 until the network converges. For all experiments, we set the parameters α1, α2, γ, λ1, λ2, λ3, and λ4 to 0.5, 1.0, 4.0, 0.005, 5.0, 0.1, and 0.7 respectively.
[0153] This embodiment is only a specific example. During specific implementation, the image size, learning rate, parameters of α1, α2, γ, λ1, λ2, λ3, λ4, etc. can also be set according to needs.
[0154] Based on the same inventive concept, this article also provides a face liveness detection device, as described in the following embodiments. Since the principle of solving the problem by the face liveness detection device is similar to that of the face liveness detection method, the implementation of the face liveness detection device can refer to the face liveness detection method, and the repeated parts will not be repeated.
[0155] Specifically, Figure 6 As shown, the face liveness detection device includes:
[0156] A storage module 610, used for storing a living body classifier established by using the living body classifier establishment method described in any of the above embodiments;
[0157] A preprocessing module 620 is used to preprocess the images collected in real time to obtain high-frequency information and enhanced images;
[0158] Extraction module 630, used to determine the camera-invariant living feature M from the high-frequency information using the first network spf ; Using the second network to determine the enhanced living feature M from the enhanced image aug ;
[0159] Prediction module 640, used to convert the camera-invariant living feature M spf And the enhanced living feature M augrespectively inputting them into the liveness classifier to obtain a first face liveness prediction result and a second face liveness prediction result;
[0160] The combination module 650 is used to combine the first face liveness prediction result and the second face liveness prediction result to obtain a face liveness prediction result.
[0161] The technical solution provided in this article can achieve the following technical effects:
[0162] (1) A face liveness detection scheme with high generalization ability based on two networks is proposed. The first network is used to extract camera-invariant liveness features, while eliminating the feature differences caused by different camera devices while extracting high-frequency liveness features. Considering that liveness features existing in other frequency domains (such as illumination, reflection) may be ignored in the first network, the second network is designed to enhance face fraud features. It improves the discriminability of features by reconstructing enhanced images from the low-frequency and high-frequency layers of the original input image.
[0163] (2) It is proposed to use an eight-directional differential filter group to extract high-frequency features and enhance living body discrimination features.
[0164] (3) The pseudo-twin neural network can be used to effectively decompose the mixed features containing liveness information and camera information, thereby decomposing the camera-invariant liveness features from the mixed features.
[0165] (4) The prediction accuracy of the model is further improved by fusing the classification results of the two branches in a weighted manner.
[0166] In one embodiment of the present invention, a computer device is also provided, such as Figure 7As shown, the computer device 702 may include one or more processors 704, such as one or more central processing units (CPUs), each of which may implement one or more hardware threads. The computer device 702 may also include any memory 706, which is used to store any kind of information such as code, settings, data, etc. In a specific embodiment, a computer program is stored on the memory 706, and the processor 704 implements the method for establishing a living body classifier described in any of the aforementioned embodiments when executing the computer program. Non-limitingly, for example, the memory 706 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory may use any technology to store information. Further, any memory may provide volatile or non-volatile retention of information. Further, any memory may represent a fixed or removable component of the computer device 702. In one case, when the processor 704 executes an associated instruction stored in any memory or combination of memories, the computer device 702 may perform any operation of the associated instruction. The computer device 702 also includes one or more drive mechanisms 708 for interacting with any storage, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.
[0167] The computer device 702 may also include an input / output module 710 (I / O) for receiving various inputs (via input device 712) and for providing various outputs (via output device 714). A specific output mechanism may include a presentation device 716 and an associated graphical user interface 718 (GUI). In other embodiments, the input / output module 710 (I / O), input device 712, and output device 714 may not be included, and the computer device 702 may be used as a computer device in a network. The computer device 702 may also include one or more network interfaces 720 for exchanging data with other devices via one or more communication links 722. One or more communication buses 724 couple the components described above together.
[0168] The communication link 722 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 722 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.
[0169] In one embodiment of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for establishing a living body classifier or the method for detecting living bodies of faces are executed.
[0170] The embodiments of the present invention also provide a computer-readable instruction, wherein when a processor executes the instruction, the program therein causes the processor to execute the steps of a living body classifier establishment method or a human face living body detection method.
[0171] It should be understood that in the various embodiments of this document, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.
[0172] It should also be understood that in the embodiments of this article, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0173] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this article.
[0174] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0175] In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.
[0176] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.
[0177] In addition, each functional unit in each embodiment of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.
[0178] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0179] Specific embodiments are used in this article to illustrate the principles and implementation methods of this article. The description of the above embodiments is only used to help understand the methods and core ideas of this article. At the same time, for general technicians in this field, according to the ideas of this article, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on this article.
Claims
1. A method for establishing a living body classifier, characterized in that: include: Preprocess the collected images to obtain high-frequency information and enhanced images; Using the first network, determine the camera-invariant living feature M from the high-frequency information spf ; Using the second network to determine the enhanced living feature M from the enhanced image aug ; Wherein, the first network includes a first sub-network and a second sub-network with the same structure and different parameters, and the first network is used to determine the camera-invariant living feature M from the high-frequency information. spf , including: inputting the high-frequency information into the first sub-network of the first network to obtain camera information M cam ; Input the high-frequency information into the second sub-network of the first network to obtain the fusion information M of the camera and the living body mix ; The camera-invariant living feature M spf And the enhanced living feature M aug Input them into the liveness classifier respectively to get the liveness prediction probability p of the first face spf And the second face live prediction probability p aug ; According to the face label, the first face live prediction result p spf And the second face live prediction result p aug , construct a binary classification focal loss function ; Using the fusion information M of the camera and the living body mix Subtract the camera information M cam , and obtain the camera-invariant living feature M spf ; The camera information M cam Input into the camera classification network to obtain the camera classification prediction probability P cam ; According to the camera label and the camera classification prediction probability P cam , construct a pixel-by-pixel multi-class focal loss function ; and / or The fusion information M of the camera and the living body mix Input into the camera classification network to obtain the camera classification prediction probability P mix ; According to the camera classification prediction probability P mix , construct a pixel-by-pixel multi-class focal loss function ; and / or The camera-invariant living feature M spf Input into the camera classification network to obtain the camera classification prediction probability P spf , according to the camera classification prediction probability P spf , construct the camera loss function ; and / or The fusion information M of the camera and the living body mix Input into the liveness classifier to obtain the liveness prediction probability p of the third face mix ; Based on the face label and the third face live prediction probability p mix , construct a binary classification focal loss function ; According to the binary focal loss function , combined with the pixel-by-pixel multi-class focal loss function , and / or pixel-wise multi-class focal loss function , and / or de-camera loss function and / or binary focal loss function , construct the target loss function; The objective loss function is used to optimize the parameters of the living body classifier, the first network and the second network.
2. The method according to claim 1, characterized in that Preprocess the collected images to obtain high-frequency information and enhanced images, including: Cutting out a facial region image from the collected image; The R, G, and B channels of the facial region image are processed using N directional differential filters to obtain 3N feature maps M diff ; According to the characteristic graph M diff , determine the high-frequency information M hf and image enhancement component I aug ; Combined with the image enhancement component I aug and the collected image to obtain an enhanced image.
3. The method according to claim 2, characterized in that According to the characteristic graph M diff , determine the high-frequency information M hf and image enhancement component I aug include: Using M*M channel convolutional neural network Conv hf For each of the 3N feature maps M diff Learn and get high-frequency information M hf ; Using three-channel convolutional neural network Conv aug For the 3N feature map M diff The feature maps of the R, G, and B channels are learned to obtain the image enhancement component I aug .
4. The method according to claim 1, characterized in that If the target loss function is composed of the two-class focal loss function , combined with the pixel-by-pixel multi-class focal loss function , pixel-by-pixel multi-class focal loss function , camera loss function Binary classification focal loss function The objective loss function is expressed by the following formula: in, is the weighting coefficient.
5. The method according to claim 1, characterized in that Determine the enhanced living feature M from the enhanced image using the second network aug ,include: The enhanced image is input into the second network to obtain the enhanced living feature M aug .
6. The method according to claim 1, characterized in that The camera-invariant living feature M spf And the enhanced living feature M aug Before inputting into the living body classifier, it also includes: The camera-invariant living feature M spf And the enhanced living feature M aug Perform average pooling.
7. A method for detecting liveness of a face, characterized in that: include: Preprocess the real-time collected images to obtain high-frequency information and enhanced images; Using the first network, determine the camera-invariant living feature M from the high-frequency information spf ; Using the second network to determine the enhanced living feature M from the enhanced image aug ; The camera-invariant living feature M spf And the enhanced living feature M aug Input them into the liveness classifier respectively to obtain the liveness prediction result of the first face and the liveness prediction result of the second face; Combining the first face liveness prediction result and the second face liveness prediction result to obtain a face liveness prediction result; Wherein, the first network, the second network and the living body classifier are trained by the living body classifier establishment method according to any one of claims 1 to 6.
8. A face liveness detection device, characterized in that: include: A storage module, used to store a living body classifier established by using the living body classifier establishment method according to any one of claims 1 to 6; A preprocessing module is used to preprocess the images collected in real time to obtain high-frequency information and enhanced images; An extraction module is used to determine the camera-invariant living feature M from the high-frequency information using the first network. spf ; Using the second network to determine the enhanced living feature M from the enhanced image aug ; A prediction module is used to transform the camera-invariant living feature M spf And the enhanced living feature M aug respectively inputting them into the liveness classifier to obtain a first face liveness prediction result and a second face liveness prediction result; A combination module is used to combine the first face liveness prediction result and the second face liveness prediction result to obtain a face liveness prediction result.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an executable computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
