Face living body detection method and device, electronic equipment and storage medium

Abstract

The present application belongs to the technical field of face living body detection, and provides a face living body detection method, device, electronic equipment and storage medium, wherein the method comprises: performing average filtering and residual processing on an original image to obtain accumulated noise features and an average filtering image; determining residual features of the original image according to the average filtering image, and performing spatial conversion on the original image to obtain I channel features and Q channel features; fusing the accumulated noise features, the residual features of the original image, the I channel features and the Q channel features to obtain fused features; performing classification training on the fused features by using an MLP-Mixer to obtain a face living body detection model; and detecting a target face image by using the face living body detection model to obtain a face living body detection result, so that the accuracy of face detection is improved and the detection cost is reduced by the technical scheme of the present application.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face liveness detection, and in particular to a face liveness detection method and device, electronic equipment and a storage medium. BACKGROUND

[0002] Face recognition is suitable for identity verification, financial security, public security and other scenarios, and the core use is to distinguish real faces from photo, video and other fraudulent attacks by analyzing the noise differences in face images, to solve the security vulnerabilities of existing face recognition systems that can be easily bypassed by fake faces, and to improve the reliability of identity verification.

[0003] Existing technologies include detection methods based on liveness clues, texture clues, 3D geometric clues and image noise analysis. The existing technologies have the following defects:

[0004] (1) The detection method based on liveness clues takes the physiological activity or dynamic feature of the real face as the core judgment basis, and is divided into two categories: motion-based method and remote photoplethysmography (rPPG) method. The motion-based method relies on video sequence input to realize dynamic feature extraction, while the rPPG method has the core logic of "dynamic feature = liveness identification", but it needs to rely on video stream input, and has limited detection capability for static fraudulent attacks, and is easily disturbed by video playback attacks;

[0005] (2) The detection method based on texture clues uses the difference between the 3D geometric structure of the real face and the 2D planar characteristics of the fake face, and is divided into 3D shape method and pseudo-depth map method. The core logic is "3D structure = liveness identification", but it needs a special 3D acquisition device, and the deployment cost is high, and the detection capability for 3D printed masks and other solid fake attacks is limited;

[0006] (3) The detection method based on image noise analysis is that fake faces will introduce additional noise (such as printed stripes, jitter, ghosting) in the imaging process, and the noise features are used to distinguish true and false: some schemes use Haar wavelet analysis to print stripe effects, or extract noise features through visual rhythm (such as extracting the gray level co-occurrence matrix (GLCM) of noise from video), but the core defect is that the noise extraction method is single, and the noise difference between real and fake faces is not fully captured, and the classifier (such as SVM) has insufficient adaptability to complex noise features, and has weak generalization ability on multiple databases. SUMMARY

[0007] To solve at least one of the technical problems in the prior art, the present application provides a face liveness detection method, device, electronic equipment and storage medium to improve the accuracy of face liveness detection.

[0008] An aspect of the present application provides a face living body detection method, comprising:

[0009] obtaining a data set, wherein the data set comprises original images;

[0010] performing average filtering and residual processing on the original images to obtain accumulated noise features and average filtering images;

[0011] determining residual features of the original images according to the average filtering images, and performing spatial conversion on the original images to obtain I channel features and Q channel features, and fusing the accumulated noise features, the residual features of the original images, the I channel features and the Q channel features to obtain fused features;

[0012] performing classification training on the fused features by using an MLP-Mixer to obtain a face living body detection model;

[0013] detecting a target face image by using the face living body detection model to obtain a face living body detection result.

[0014] According to the face living body detection method, wherein the obtaining of the data set further comprises:

[0015] annotating the original images according to image types, wherein the image types comprise living body images and fake images;

[0016] performing linear filtering processing on pictures of the data set by using a linear filter and preset MLP-Mixer parameters to obtain linear filtering images, wherein the preset MLP-Mixer parameters comprise a block number, an output channel number, a hidden layer dimension, a training round, a weight attenuation coefficient and a learning rate;

[0017] preprocessing the linear filtering images by using an MTCNN algorithm, wherein the MTCNN algorithm sequentially performs face region positioning, rejection processing of lower than a preset confidence, key point affine transformation and ROI alignment on the linear filtering images.

[0018] According to the face living body detection method, wherein the performing of the average filtering and the residual processing on the original images to obtain the accumulated noise features and the average filtering images comprises:

[0019] performing pixel checking on the original images according to the linear filtering images;

[0020] performing average filtering processing on the original images by using an average filter for a preset number of rounds to obtain average filtering images;

[0021] obtaining each round of residual of adjacent two times of average filtering by using pixel-level subtraction on the average filtering images, and cropping the linear filtering images according to each round of residual;

[0022] The average filtered image and each round residual are stored according to the round, and the cumulative noise feature is obtained by performing mean filtering smoothing processing on the sum of the normalized each round residual.

[0023] According to the face living body detection method, the residual feature of the original image is determined according to the average filtered image, the original image is spatially converted to obtain the I channel feature and the Q channel feature, and the cumulative noise feature, the residual feature of the original image, the I channel feature and the Q channel feature are fused to obtain the fusion feature, including:

[0024] The calculation formula of the residual feature of the original image according to the average filtered image is:

[0025] ;

[0026] Wherein, is the residual feature of the original image, is the original image, is the average filtered image, is the value of the preset round;

[0027] The face ROI of the original image is converted from the RGB space to the YIQ space according to the standard television system parameters to obtain the I channel feature and the Q channel feature.

[0028] The cumulative noise feature, the residual feature of the original image, the I channel feature and the Q channel feature are sequentially spliced according to the channel dimension to obtain the fusion feature.

[0029] According to the face living body detection method, the fusion feature is classified and trained by using MLP-Mixer to obtain a face living body detection model, including:

[0030] The fusion feature is divided into non-overlapping blocks, and the overlapping blocks are flattened and L2 normalized to obtain a block sequence.

[0031] The first fully connected network and the random drop strategy are used to perform cross-channel fusion on the block sequence to obtain a fusion representation.

[0032] The channel average value of the fusion representation is calculated according to the dimension of the block sequence to obtain a global feature, and the global feature is processed by the second fully connected network to map the feature to the label to obtain a classification result, wherein the classification result includes a living probability and a fake probability.

[0033] According to the classification result, the cross-entropy loss function, the Adam optimizer and the early stop strategy are used for training to obtain the face living body detection model, wherein the cross-entropy loss function is:

[0034] ;

[0035] wherein, is a cross-entropy loss function, is a training sample serial number identifier, is a total number of training samples, represents a real label of the th training sample, is a live probability of the th training sample, is a fake probability of the th training sample.

[0036] According to the face live body detection method, wherein the face live body detection model is used to detect the target face image to obtain a face live body detection result, comprising:

[0037] The image acquisition device is used to acquire the target face image according to preset acquisition parameters, wherein the target face image is a two-dimensional image.

[0038] The MTCNN algorithm is used to locate the face region of the target face image, the Laplacian variance method is used to calculate the sharpness of the face region, the target face image is abnormally judged and pretreated according to the sharpness, and a pretreated image is obtained.

[0039] The face live body detection model is used to detect the pretreated image to obtain a face live body detection result.

[0040] According to the face live body detection method, wherein the method further comprises:

[0041] The detection data of the face live body detection model in a preset period is acquired in real time, and a model performance evaluation report is generated according to the detection data;

[0042] If the error rate of the core index of the model performance evaluation report is higher than a preset error rate, the model is retrained by adjusting the iteration number of the linear filter and supplementing the spoofing attack data of the data set;

[0043] The optimization process data of the face live body detection model is recorded, and the face live body detection model is updated in a closed loop iteration manner according to the optimization process data.

[0044] Another aspect of the present application provides a face live body detection device, comprising:

[0045] A first module is used to acquire a data set, wherein the data set comprises an original image.

[0046] A second module is used to perform average filtering and residual processing on the original image to obtain accumulated noise features and an average filtered image.

[0047] a third module configured to determine residual features of the original image according to the average filtered image, perform spatial conversion on the original image to obtain I-channel features and Q-channel features, and fuse the accumulated noise features, the residual features of the original image, the I-channel features and the Q-channel features to obtain fused features;

[0048] a fourth module configured to perform classification training on the fused features by using an MLP-Mixer to obtain a face liveness detection model;

[0049] a fifth module configured to perform detection on a target face image by using the face liveness detection model to obtain a face liveness detection result.

[0050] Another aspect of the embodiment of the present application provides an electronic device including a processor and a memory;

[0051] The memory is configured to store a program.

[0052] The processor executes the program to implement the method as described above.

[0053] The embodiment of the present application also discloses a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to perform the method as described above.

[0054] The present application has the following beneficial effects: the linear filter iterative residual noise extraction mode iteratively accumulates the noise difference between real and fake faces, improves the noise feature discrimination, fuses noise, residual and YIQ color multi-dimensional features, compensates for the limitations of single features, improves the robustness of multiple types of attacks such as printing, video and 3D mask, replaces the traditional SVM / CNN with MLP-Mixer, captures the global correlation of noise features through Token / Channel Mixing, reduces the error rate of face liveness detection, only needs a single 2D image input, adapts to ordinary image acquisition devices, does not need 3D hardware or video stream, and reduces the detection cost. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a face liveness detection flowchart of the embodiment of the present application.

[0056] Figure 2 is an original image processing flowchart of the embodiment of the present application.

[0057] Figure 3Figure 1 is a comparison chart of real faces and fake faces with different noises according to an embodiment of the present application, wherein (a) is a real face image; (b) is a dithering noise image; (c) is a stripe effect noise image; (d) is a mirror reflection noise image; and (e) is a distortion noise image.

[0058] Figure 4 Figure 2 is a schematic diagram of an average filtering and residual processing flow according to an embodiment of the present application.

[0059] Figure 5 Figure 3 is an example of a live face and a fake face after iterative average filtering according to an embodiment of the present application, wherein (a) is an initial original image without filtering; (b) is a residual image after 1 iteration; and (c) is a residual image after 50 iterations.

[0060] Figure 6 Figure 4 is a schematic diagram of a classification training flow based on MLP-Mixer according to an embodiment of the present application.

[0061] Figure 7 Figure 5 is a framework diagram of MLP-Mixer according to an embodiment of the present application.

[0062] Figure 8 Figure 6 is a schematic diagram of model optimization according to an embodiment of the present application.

[0063] Figure 9 Figure 7 is a schematic diagram of a face liveness detection device according to an embodiment of the present application. DETAILED DESCRIPTION

[0064] Embodiments of the present application are described in detail below with reference to the accompanying drawings. The same or similar components are denoted by the same or similar reference numerals throughout the drawings. In the following description, the suffixes "module", "part" or "unit" are used only in order to facilitate explanation of the present application, and are not intended to have a specific meaning or function. Therefore, "module", "part" or "unit" can be mixedly used. "First", "second", and so on are used only to distinguish technical features for the purpose of explanation, and cannot be understood to indicate or imply relative importance or implicitly indicate the number of indicated technical features or the order of the indicated technical features. In the following description, the consecutive numbers of the method steps are for the convenience of review and understanding, and adjusting the implementation order between the steps does not affect the technical effects achieved by the technical solution of the present application in combination with the overall technical solution of the present application and the logical relationship between the steps. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0065] Reference Figure 1 The face liveness detection flowchart shown includes but is not limited to steps S100-S500:

[0066] S100, obtaining a data set, wherein the data set comprises an original image.

[0067] In some embodiments, referring to the original image processing flowchart shown, which includes but is not limited to steps S110-S130: Figure 2

[0068] S110, labeling the original image according to image types, wherein the image types include live images and fake images;

[0069] S120, performing linear filtering processing on the labeled original image using a linear filter and preset MLP-Mixer parameters to obtain a linear filtering image, wherein the preset MLP-Mixer parameters include block number, output channel number, hidden layer dimension, training round, weight decay coefficient, and learning rate;

[0070] S130, preprocessing the linear filtering image using an MTCNN algorithm, wherein the MTCNN algorithm sequentially performs face region positioning, rejection processing below a preset confidence level, key point affine transformation, and ROI alignment on the linear filtering image.

[0071] In some embodiments, the NUAA photo spoofing database is imported, which contains 12612 images of 15 subjects, including 5105 live images and 7509 fake images, and the fake types include printed photos and screen display images. The database is divided into a training set (3573 live images and 5256 fake images) and a test set (1532 live images and 2253 fake images) in a 7:3 ratio. The CASIA face anti-spoofing database is imported, which contains 600 videos of 50 subjects, including 3 real videos and 9 fake videos (print and video playback attack) for each subject. 10 key frames are extracted from each video, a total of 6000 frames, and the training set (4200 frames) and the test set (1800 frames) are divided in a 7:3 ratio. All images are labeled as live (label 1) or fake (label 0). During the labeling process, the attack type description of the original database is referred to to ensure that the labeling is consistent with the actual spoofing type and avoid label bias affecting model training. After labeling is completed, the data set is de-duplicated to delete duplicate or highly similar images and ensure sample diversity.

[0072] In some embodiments, the linear filter parameters are: window size 3x3, iteration number K=90, and the filtering formula is:

[0073] ;

[0074] The filtering formula indicates that after the i-th round of linear filtering, the pixel value of the image at position (x, y) is ​, which is equal to the average value of all pixel values in the 3x3 neighborhood centered at (x, y) in the image after the i-1th round of filtering. represents the two-dimensional coordinates of a certain pixel in the image, and a and b represent the offset of the 3x3 filter window.

[0075] During the filtering process, the image after each round of filtering is temporarily stored in the local cache and named in the format of filtering round-image identifier, which facilitates subsequent tracing and troubleshooting.

[0076] In some embodiments, the MLP-Mixer parameters are as follows:

[0077] Number of blocks , input channel number C=6, hidden layer dimension 512, training round 100 rounds, weight decay coefficient 1e-5, learning rate using cosine annealing strategy, initial learning rate 1e-4, decaying to 80% of the current value every 20 rounds; set the evaluation indicators as accuracy, equal error rate, area under the ROC curve, false acceptance rate, false rejection rate, and half total error rate, calculate the evaluation indicators every 10 rounds of training, and generate a performance report at each stage.

[0078] In some embodiments, the MTCNN algorithm is used to perform face detection on the input image, the confidence threshold is set to 0.8, and after locating the face region, images without faces, multiple faces, and confidence lower than the threshold are removed; affine transformation is performed based on the eye and nose key points, and the triangle formed by the left eye, right eye, and nose tip is used as the reference to align the face to the 224x224 pixel ROI. During the alignment process, the face center is kept coincident with the ROI center, and the pose difference is eliminated. The ROI pixel value is normalized according to the formula:

[0079] ;

[0080] The formula is the image pixel normalization calculation formula, which is used to map the original image pixel value uniformly to the interval [0, 1], where is the pixel value of the coordinate (x, y) of the original image, is the minimum pixel value of the entire original image, is the maximum pixel value of the entire original image, is the pixel value of the normalized coordinate (x, y).

[0081] After normalization, the pixel value distribution is checked to ensure that there are no outliers outside the range [0, 1]. All standardized data is stored in the system knowledge base in the format of "dataset type-sample type-sequence number", and the knowledge base uses a distributed storage architecture to support parallel reading of multiple modules, serving as the input source for subsequent modules.

[0082] Reference Figure 3 , Figure 3, real face and fake face with different noise, where (a) is a real face image; (b) is a dithering noise image; (c) is a stripe effect noise image; (d) is a mirror reflection noise image; (e) is a distortion noise image, where (b)-(e) are fake face images including typical noise, by Figure 3 It can be seen that fake faces will introduce unique noise due to the resampling / printing process, while real faces do not have such noise.

[0083] According to Figure 3 It can be seen that as the number of iterations increases, the noise difference between real and fake faces will continue to accumulate, providing more discriminative features for subsequent classification.

[0084] S200, average filtering and residual processing are performed on the original image to obtain accumulated noise features and an average filtered image.

[0085] In some embodiments, referring to Figure 4 the average filtering and residual processing flowchart shown, which includes but is not limited to steps S210-S240:

[0086] S210, pixel checking is performed on the original image according to the linear filtered image.

[0087] In some embodiments, the face ROI image after preprocessing according to the above embodiments Let the initial unfiltered image be ; pixel value checking is performed on to ensure that the image size is 224x224 and the number of channels is 3. If the size does not match or the channel is abnormal, return to step S1.3 for reprocessing to ensure that the initial image retains complete original details, providing a reference for subsequent noise extraction.

[0088] S220, average filtering is performed on the original image using an average filter for a preset number of rounds to obtain an average filtered image.

[0089] S230, pixel-level subtraction is performed on the average filtered image to obtain the residual of each round of adjacent average filtering, and the linear filtered image is cropped according to the residual of each round.

[0090] In some embodiments, an iteration operation is performed for i=1 to 90 rounds: the filtered image of the previous round is input into a 3x3 average filter according to the formula

[0091] ;

[0092] to obtain the current round filtered image, where is the filtered image obtained after the i-th iteration, is the filtered image obtained after the (i-1)-th iteration, the filtered image after the i-th iteration, the average filtering operation representing a 3x3 window.

[0093] After the filtering is completed, the pixel difference value of and is obtained by pixel-level subtraction to obtain the i-th round residual noise , and the positive and negative residual values are retained; after each round of residual calculation, the range clipping is performed on to limit the pixel value in the interval [-0.5, 0.5], so as to avoid the interference of extreme values on the subsequent feature accumulation; and and are stored according to the round, and the complete data chain of the iteration process is established.

[0094] In S240, the average filtered image and each round of residual are stored according to the round, and after the normalization of each round of residual, the sum is calculated, and then the mean filtering smoothing processing is performed to obtain the cumulative noise feature.

[0095] In some embodiments, the sum of 90 rounds of residual noise images is calculated to obtain the cumulative noise feature :

[0096] ;

[0097] Before the sum is calculated, the normalization processing is performed on each round of , according to the formula

[0098] ;

[0099] The each round of residual noise is unified to the interval [0, 1] to ensure the weight balance of each round of noise feature, wherein is the residual noise image obtained after the i-th round of average filtering, , are the minimum and maximum pixel values of the i-th round of residual noise image respectively, is the normalized i-th round of residual noise; after the accumulation is completed, the size is consistent with the face ROI, the mean filtering smoothing processing is performed on it to eliminate the local extreme noise points, the noise difference between the living body and the fake face is amplified through multiple rounds of accumulation, and high-discrimination features are provided for subsequent classification.

[0100] In some embodiments, reference is made to Figure 5The comparison figure of the real face and the fake face with different noises is shown, wherein (a) is the input real face image, that is, the initial unfiltered face image, and the visual difference between the live face and the fake face is not significant; (b) is the residual image after 1 iteration, and the difference between the residual noise image calculated after 1 round of linear filtering is small; (c) is the residual image after 50 iterations, and the residual noise is accumulated after 50 rounds of linear filtering, the live face can still retain more details, the fake face loses more details, and the difference between the two is significant.

[0101] S300, determining a residual feature of the original image according to the average filtered image, and performing spatial conversion on the original image to obtain an I channel feature and a Q channel feature, and fusing the accumulated noise feature, the residual feature of the original image, the I channel feature and the Q channel feature to obtain a fused feature.

[0102] In some embodiments, the calculation formula for determining the residual feature of the original image according to the average filtered image is:

[0103] ;

[0104] wherein, is the residual feature of the original image, is the original image, is the average filtered image, is the value of the preset round.

[0105] For example, the image filtered for the 90th round , the residual of the original image is calculated according to the formula ; after the calculation is completed, edge enhancement processing is performed on , the edge information of the residual image is extracted by using a Sobel operator to enhance the texture details; the residual feature after edge enhancement is scaled to 224x224 size to ensure consistency with other feature sizes, and the residual feature can assist the noise feature to distinguish the texture difference between the live face and the fake face.

[0106] In some embodiments, the face ROI of the original image is converted from the RGB space to the YIQ space according to the standard television system parameters to obtain the I channel feature and the Q channel feature.

[0107] For example, the original face ROI is converted from the RGB space to the YIQ space, and the conversion formula is set according to the standard television system parameters, wherein the Y channel calculation adopts the luminance formula , the I channel is , and the Q channel is ; only the I channel ( ) and the Q channel ( ) features are extracted, and the histogram equalization is performed on the two channel features respectively to enhance the color contrast; the equalized With Scaling to 224x224 size, this feature can further improve the classification robustness.

[0108] In some embodiments, the cumulative noise feature, the residual feature of the original image, the I channel feature and the Q channel feature are sequentially spliced in the channel dimension to obtain a fusion feature.

[0109] Exemplarily, the cumulative noise feature , the residual feature , the I channel feature and the Q channel feature are spliced in the channel dimension, and the splicing order is - - - , to obtain a fusion feature ; after splicing, the feature dimension is checked to ensure that the number of channels is 4 and the size is 224x224, and if the dimension is not consistent, the previous step is returned for reprocessing; the is normalized according to the formula

[0110] ;

[0111] The feature mean is normalized to 0 and the standard deviation is normalized to 1, which compensates for the insufficient adaptation of a single feature to complex spoofing attacks through multi-dimensional information complementation.

[0112] S400, using MLP-Mixer to classify and train the fusion feature to obtain a face liveness detection model.

[0113] In some embodiments, referring to the MLP-Mixer-based classification training process diagram shown in Figure 6 , which includes but is not limited to steps S410-S440:

[0114] S410, dividing the fusion feature into non-overlapping blocks, and performing flattening and L2 normalization on overlapping blocks to obtain a block sequence.

[0115] Exemplarily, the fusion feature is divided into 196 non-overlapping blocks, and the block size is 16x16x4. During the division process, starting from the top left corner of the image, the blocks are arranged in row priority order to avoid block overlap or omission; each block is flattened into a 1x1024 vector to form a block sequence ; after flattening, L2 normalization is performed on each vector according to the formula

[0116] ;

[0117] The formula is the L2 normalization calculation formula of the vector. Wherein, is a one-dimensional vector after flattening the fused feature blocks, is L2 norm of is the normalized vector. Ensure that the vector length is 1, realize the structured input and normalization of features.

[0118] S420, perform cross-channel fusion on the block sequence using the first fully connected network and the random dropout strategy to obtain the fusion representation.

[0119] S430, calculate the channel average value of the fusion representation according to the dimension of the block sequence to obtain the global feature, and perform feature and label mapping processing on the global feature through the second fully connected network to obtain the classification result, wherein the classification result includes the living probability and the fake probability.

[0120] In some embodiments, cross-block fusion is performed, and the block sequence is first fused according to the formula

[0121] ;

[0122] Layer normalization is performed, and the formula is the layer normalization calculation formula of the feature, wherein X is the block sequence feature matrix in the MLP-Mixer, is the mean of X, is the variance of X, is the minimum value, is the normalized feature matrix; convert to , input a 2-layer fully connected network (first fully connected network), the first hidden layer dimension is 2048, the activation function is GELU, the second hidden layer dimension is 1024, the activation function is GELU, and the output is ; convert back to , check the dimension consistency after transposition, and ensure that it matches the input block sequence dimension.

[0123] In some embodiments, cross-channel fusion is performed, and the layer normalization is performed again, and a 2-layer fully connected network (second fully connected network) is input, the network structure is consistent with the fully connected network of TokenMixing, and the output is ; a dropout strategy is used in the fusion process, the dropout probability is set to 0.1, and the model is prevented from overfitting to capture the correlation between different channels.

[0124] S440, according to the classification result, use the cross-entropy loss function, Adam optimizer and early stopping strategy to train to obtain the face living body detection model.

[0125] In some embodiments, the cross-entropy loss function is:

[0126] ;

[0127] wherein, is a cross-entropy loss function, is a training sample serial number, is a total number of training samples, represents the real label of the th training sample, is 0 or 1, indicating that the training sample is a living face ( ) or a fake image ( ); is the living probability of the th training sample, is the fake probability of the th training sample.

[0128] For example, the optimizer uses Adam, the learning rate is 1e-4, the weight decay is 1e-5, the gradient clipping threshold is set to 1.0 to prevent gradient explosion; during training, an early stopping strategy is used, if the validation set ACC does not improve for 10 consecutive rounds, the training is terminated in advance; after each round, the performance is evaluated on the validation set, and the model with the highest validation set ACC (recorded as ) is saved; the model file is named according to "training date-validation set ACC", and stored in the system model warehouse; the warehouse uses a version control mechanism to support model backtracking and comparison.

[0129] In some embodiments, referring to the MLP-Mixer framework diagram shown in Figure 7 , different color blocks represent different sampling channels, Figure 7 including the internal structure of a single MixerLayer in MLP-Mixer, mainly including TokenMixingMLP and ChannelMixingMLP two types of sub-modules, which cooperatively complete global feature fusion:

[0130] (1) Feature blocking and initial transformation: first, the noise image extracted by iteration is divided into several non-overlapping image blocks, and each block is subjected to feature dimension transformation by a fully connected layer to form a block sequence feature (dimension PxC, P is the number of blocks, and C is the number of channels);

[0131] (2) TokenMixing (cross-block fusion): layer normalization is performed on the block sequence feature to avoid the influence of numerical fluctuations; the PxC feature is transposed to CXP, so that the channel dimension is in the front; the P blocks of C channels are fused by an MLP with shared parameters to capture the global correlation between different blocks (such as the noise distribution relationship of different regions of a face); the processed CXP feature is converted back to PxC to maintain dimensional consistency.

[0132] (3) ChannelMixing (cross-channel fusion): PxC features output by TokenMixing are normalized again; P blocks of C channel features are fused by an MLP to capture the correlation between different channels.

[0133] (4) Multi-layer stacking and classification: after stacking multiple MixerLayers (8 layers in the embodiment), the block sequence features are compressed into global features by global average pooling, and finally the classification probability of live / dead is output by a fully connected layer.

[0134] S500, detecting the target face image by the face live detection model to obtain a face live detection result.

[0135] In some embodiments, the target face image is a two-dimensional image.

[0136] For example, the face image of the object to be detected is collected by a common 2D camera, the camera resolution is set to 480x640, the shooting distance is controlled to be 0.5-1.5 meters during collection to avoid direct light or strong light; if the output resolution of the collection device does not match, a bilinear interpolation algorithm is used to automatically scale to 480x640, the aspect ratio of the image is kept unchanged during the scaling process to avoid face deformation; the original image collected is temporarily stored in a temporary cache, the cache validity period is set to 5 minutes, and the cache is automatically cleaned up when it expires to ensure that the input format is consistent with the training sample.

[0137] In some embodiments, the face region of the target face image is located by using the MTCNN algorithm, the sharpness of the face region is calculated by using the Laplacian variance method, and the target face image is abnormally judged and preprocessed according to the sharpness to obtain a preprocessed image.

[0138] For example, when the face region is located by using the MTCNN algorithm and no face is detected or multiple faces are detected, the output is “no valid face detected, please re-collect”; the image sharpness is calculated by using the Laplacian variance method, and the formula is:

[0139] ;

[0140] If the sharpness is less than 0.8, the output is “image is blurred, please adjust the shooting environment”; the face region and the background are segmented by using a semantic segmentation algorithm, the face occlusion area ratio is calculated, and if the occlusion area is greater than 30%, the output is “too much face occlusion, please avoid occlusion and re-collect”. Through the determined image, affine transformation is performed based on the eye and nose key points to align to a 224x224 pixel ROI, which is normalized to the [0, 1] interval according to the formula to obtain the preprocessed image ; and Quality inspection is performed to ensure that there are no pixel value abnormalities, size inconsistencies, etc., and if there are abnormalities, the preprocessing is re-executed.

[0141] In some embodiments, the preprocessed image is detected by a face liveness detection model to obtain a face liveness detection result.

[0142] For example, the above-mentioned preprocessing and feature extraction are repeated, and the preprocessed image is detected by a face liveness detection model to obtain a face liveness detection result. 90 rounds of 3x3 average filter iterations are performed, and after each iteration, the filtered image and residual noise are stored, and finally the cumulative noise feature is obtained. The residual feature is calculated. The image is converted from RGB space to YIQ space, and I / Q channel features are extracted. , , The above-mentioned features are concatenated in the channel dimension to obtain the fusion feature to be detected. After concatenation, the feature dimension is verified to ensure that the channel number is 4 and the size is 224x224, and if the dimension is abnormal, the extraction is returned.

[0143] In some embodiments, inference and result feedback are also included:

[0144] Fusion_Feat(test) is input into Best_Model, the model inference uses batch processing mode, and the batch size for a single inference is set to 8 to improve processing efficiency; P_(live(test)) and P_(spoof(test)) are output, if P_(live(test))≥0.95, the result is determined to be "live"; if P_(spoof(test))≥0.95, the result is determined to be "fake"; if both are <0.95, output "the recognition result is uncertain, please reacquire".

[0145] The determination result, the image to be detected, and the feature extraction log are stored in the detection result database in association, and the log includes information such as acquisition time, device number, preprocessing time consumption, inference time consumption, etc.; if it is a terminal application scenario, the determination result is output to the terminal in real time through an encrypted communication link, and the terminal triggers subsequent actions according to the result, such as unlocking the access control, rejecting access, or alarm prompt, and operation records are generated after the actions are executed and are returned to the result database for archiving.

[0146] Embodiments of the present application also include iterative optimization of the model, as shown in the model optimization process schematic diagram, which includes but is not limited to steps S610-S630: Figure 8

[0147] ​S610, real-time collection of detection data of the face liveness detection model in a preset period, generation of a model performance evaluation report according to the detection data, including performance evaluation report generation, optimization direction and parameter adjustment, and model updating and closed-loop iteration.

[0148] For example, the performance evaluation report generation includes statistics of real-time detection data per month, generation of a model performance evaluation report; the report contains the total number of detections, the number of valid detections, the number of re-sampling times, and the statistical values and trend charts of core indicators such as ACC, EER, FAR, FRR, and HTER in the period; the number and reasons of re-sampling prompts are analyzed, and the proportions of “blur”, “occlusion”, “no face” and other categories are counted to locate the key factors affecting the detection efficiency; the evaluation report is generated in PDF format and stored in the system knowledge base and the management terminal for technical personnel to review and analyze.

[0149] S620, if the error rate of the core indicators of the model performance evaluation report is higher than the preset error rate, retraining is performed by adjusting the iteration number of the linear filter and supplementing the spoofing attack data of the data set.

[0150] In some embodiments, if EER>0.05%, adjust the iteration number of the linear filter, with an adjustment amplitude of 5 each time, such as from 90 to 95, re-execute steps S2-S4 to train a new model, and compare the model performance under different iteration numbers during the training process to determine the optimal iteration number; if ACC<99.9%, supplement new spoofing attack data, including 3D printed masks, hyper-real silicone masks, dynamic screen displays, and other new attack images, and perform double cross-labeling on the new data according to the labeling rules of step S1.1 to ensure that the label accuracy is >99.5%. If the amount of new data is more than 10% of the original training set, use the incremental training mode, freeze the first 50 layers of the MLP-Mixer, and only fine-tune the last 30 layers and the classification head, with a learning rate of 5e-5 and a training round of 50 rounds; if the amount of new data is small, use a data augmentation strategy to expand the samples, and the augmentation methods include random rotation, horizontal flip, and brightness adjustment to avoid overfitting and resource waste caused by full training.

[0151] S630, record the optimization process data of the face liveness detection model, and perform closed-loop iterative updating of the face liveness detection model through the optimization process data.

[0152] Exemplarily, the new model, the evaluation report and the optimized parameters are stored in the system knowledge base, the knowledge base automatically associates historical data to form a model optimization track; the performance of the new model and the historical optimal model is compared, if the ACC of the new model is improved by greater than or equal to 0.1% and the EER is reduced by greater than or equal to 0.01%, the new model is set as a default detection model; returning to step S1, the system is reinitialized based on the updated parameters, the update content includes data set supplement, parameter adjustment, preprocessing rule optimization, the next round of 'initialization-detection-optimization' closed loop is started, and the adaptation ability and detection precision of the model to new type of fraud attacks are continuously improved.

[0153] Figure 9 is a schematic diagram of a face liveness detection device according to an embodiment of the present application. The device includes a first module 910, a second module 920, a third module 930, a fourth module 940 and a fifth module 950.

[0154] The first module is configured to obtain a data set, wherein the data set includes an original image; the second module is configured to perform average filtering and residual processing on the original image to obtain accumulated noise features and an average filtered image; the third module is configured to determine residual features of the original image according to the average filtered image, and perform spatial conversion on the original image to obtain I-channel features and Q-channel features, and fuse the accumulated noise features, the residual features of the original image, the I-channel features and the Q-channel features to obtain fused features; the fourth module is configured to perform classification training on the fused features using an MLP-Mixer to obtain a face liveness detection model; and the fifth module is configured to detect a target face image through the face liveness detection model to obtain a face liveness detection result.

[0155] Exemplarily, under the cooperation of the first module, the second module, the third module, the fourth module and the fifth module in the device, the device according to the embodiment can implement any one of the face liveness detection methods described above, that is, a data set is obtained, wherein the data set includes an original image; average filtering and residual processing are performed on the original image to obtain accumulated noise features and an average filtered image; residual features of the original image are determined according to the average filtered image, and spatial conversion is performed on the original image to obtain I-channel features and Q-channel features, and the accumulated noise features, the residual features of the original image, the I-channel features and the Q-channel features are fused to obtain fused features; classification training is performed on the fused features using an MLP-Mixer to obtain a face liveness detection model; and a target face image is detected through the face liveness detection model to obtain a face liveness detection result.

[0156] An electronic device according to an embodiment of the present application includes a processor and a memory;

[0157] The memory stores a program;

[0158] The processor executes the program to perform the aforementioned face liveness detection method; the electronic device has the function of carrying and running the software system of the face liveness detection provided by the embodiment of the present application, for example, a personal computer, a mini computer, a mainframe, a workstation, a network or a distributed computing environment, a single or integrated computer platform, or communication with a charged particle tool or other imaging device, and the like.

[0159] The embodiment of the present application also provides a computer readable storage medium, the storage medium stores a program, and the program is executed by a processor to realize the face liveness detection method as described above.

[0160] In some alternative embodiments, the functions / operations mentioned in the block diagram can not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially simultaneously with each other, or the blocks can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flowcharts of the present application are provided by way of example, with the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented by the embodiments of the present application. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of larger operations are independently executed.

[0161] The embodiment of the present application also discloses a computer program product or a computer program, the computer program product or the computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the aforementioned face liveness detection method.

[0162] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and / or features described can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It can also be understood that a detailed discussion of the actual implementation of each module is unnecessary for an understanding of the present application. Rather, given the properties, functions and internal relationships of the various functional modules disclosed in the embodiments of the present application, the actual implementation of the module will be within the routine skill of an engineer, given the properties, functions and internal relationships of the various functional modules disclosed in the embodiments of the present application. Therefore, those skilled in the art can implement the present application as set forth in the claims without undue experimentation using ordinary skill. It can also be understood that the disclosed specific concepts are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the appended claims and their equivalents.

[0163] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0164] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered a list of executable instructions for implementing logic functions, and can be specifically embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or in conjunction with these instructions execution systems, apparatuses, or devices. For the purpose of this specification, "computer-readable medium" can be any device that can include, store, communicate, propagate, or transport programs for use by an instruction execution system, apparatus, or device, or in conjunction with these instruction execution systems, apparatuses, or devices.

[0165] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CD ROM). In addition, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by editing, interpreting, or otherwise processing, if necessary, in other suitable ways, to be electronically obtained, and then stored in the computer memory.

[0166] It should be understood that aspects of the application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware which is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, can be used: a combination of discrete logic circuits having logic gates for implementing logic functions upon an application of data signals, application specific integrated circuits having logic gates, field programmable gate arrays (FPGA), or others.

[0167] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or more embodiments or examples.

[0168] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, alternatives, and variations can be made to the embodiments without departing from the principles and spirit of the application, and the scope of the present application is defined by the appended claims and their equivalents.

[0169] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are included in the scope defined by the claims of the present application.

Claims

1. A method for detecting human face liveness, characterized in that, include: Obtain the dataset, which includes the original images; The original image is subjected to average filtering and residual processing to obtain cumulative noise features and average filtered image; The residual features of the original image are determined based on the average filtered image, and the original image is spatially transformed to obtain I-channel features and Q-channel features. The cumulative noise features, the residual features of the original image, the I-channel features and the Q-channel features are fused to obtain the fused features. The fused features were classified and trained using MLP-Mixer to obtain a face liveness detection model. The face liveness detection model is used to detect the target face image to obtain the face liveness detection result; The residual features of the original image are determined based on the average filtered image, and the original image is spatially transformed to obtain I-channel features and Q-channel features. The accumulated noise features, the residual features of the original image, the I-channel features, and the Q-channel features are fused to obtain fused features, including: The formula for calculating the residual features of the original image based on the average filtered image is as follows: in, The residual features of the original image, For the original image, For the average filtered image, This is the value for the preset round; The face ROI of the original image is converted from RGB space to YIQ space according to standard television system parameters to obtain I-channel features and Q-channel features; The cumulative noise features, residual features of the original image, I-channel features, and Q-channel features are sequentially concatenated along the channel dimension to obtain the fused features; The method of using MLP-Mixer to classify and train the fused features to obtain a face liveness detection model includes: The fused features are divided into non-overlapping blocks, and the overlapping blocks are flattened and L2 normalized to obtain a block sequence. A first fully connected network and a random dropout strategy are used to perform cross-channel fusion on the block sequence to obtain the fused representation; The fusion representation is calculated according to the dimension of the block sequence to obtain the global features. The global features are then mapped to labels through a second fully connected network to obtain the classification results, which include the liveness probability and the forgery probability. Based on the classification results, the face liveness detection model is trained using the cross-entropy loss function, the Adam optimizer, and an early stopping strategy. The cross-entropy loss function is: in, Let cross-entropy be the loss function. This serves as the identifier for the training sample sequence. The total number of training samples, Indicates the first The true labels of each training sample For the first The liveness probability of each training sample. For the first The probability of a training sample being faked.

2. The face liveness detection method according to claim 1, characterized in that, The acquisition of the dataset also includes: The original images are labeled according to their image types, which include live images and forged images; The labeled original image is linearly filtered using a linear filter and preset MLP-Mixer parameters to obtain a linearly filtered image. The preset MLP-Mixer parameters include the number of blocks, the number of output channels, the dimension of the hidden layer, the number of training epochs, the weight decay coefficient, and the learning rate. The linearly filtered image is preprocessed using the MTCNN algorithm, which sequentially performs face region localization, removal of images with confidence levels below a preset threshold, affine transformation of key points, and ROI alignment on the linearly filtered image.

3. The face liveness detection method according to claim 2, characterized in that, The step of performing average filtering and residual processing on the original image to obtain cumulative noise features and an average filtered image includes: Perform pixel verification on the original image based on the linearly filtered image; The original image is subjected to an average filter for a preset number of rounds of average filtering to obtain an average filtered image. The average filtered image is subtracted at the pixel level to obtain the residual of each round of average filtering between two adjacent rounds, and the linearly filtered image is cropped based on the residual of each round. The average filtered image and the residuals of each round are stored according to the round number. The residuals of each round are normalized and summed, and then mean filtering is performed to smooth them, thus obtaining the cumulative noise features.

4. The face liveness detection method according to claim 1, characterized in that, The step of detecting the target face image using the face liveness detection model to obtain the face liveness detection result includes: The target face image is acquired by an image acquisition device according to preset acquisition parameters, wherein the target face image is a two-dimensional image; The MTCNN algorithm is used to locate the face region in the target face image, and the Laplacian variance method is used to calculate the sharpness of the face region. Based on the sharpness, anomaly detection and preprocessing are performed on the target face image to obtain a preprocessed image. The preprocessed image is detected using the aforementioned face liveness detection model to obtain face liveness detection results.

5. The face liveness detection method according to claim 4, characterized in that, The method further includes: Real-time acquisition of face liveness detection model detection data within a preset period, and generation of model performance evaluation report based on the detection data; If the error rate of the core metrics in the model performance evaluation report is higher than the preset error rate, retraining is performed by adjusting the number of iterations of the linear filter and supplementing the dataset with deceptive attack data. Record the optimization process data of the face liveness detection model, and use the optimization process data to perform closed-loop iterative updates to the face liveness detection model.

6. A face liveness detection device, characterized in that, include: The first module is used to acquire the dataset, which includes the original images; The second module is used to perform average filtering and residual processing on the original image to obtain cumulative noise features and average filtered image; The third module is used to determine the residual features of the original image based on the average filtered image, and to perform spatial transformation on the original image to obtain I-channel features and Q-channel features. The cumulative noise features, residual features of the original image, I-channel features and Q-channel features are fused to obtain fused features. The fourth module is used to classify and train the fused features using MLP-Mixer to obtain a face liveness detection model; The fifth module is used to detect the target face image through the face liveness detection model and obtain the face liveness detection result; The residual features of the original image are determined based on the average filtered image, and the original image is spatially transformed to obtain I-channel features and Q-channel features. The accumulated noise features, the residual features of the original image, the I-channel features, and the Q-channel features are fused to obtain fused features, including: The formula for calculating the residual features of the original image based on the average filtered image is as follows: in, The residual features of the original image, For the original image, For the average filtered image, This is the value for the preset round; The face ROI of the original image is converted from RGB space to YIQ space according to standard television system parameters to obtain I-channel features and Q-channel features; The cumulative noise features, residual features of the original image, I-channel features, and Q-channel features are sequentially concatenated along the channel dimension to obtain the fused features; The method of using MLP-Mixer to classify and train the fused features to obtain a face liveness detection model includes: The fused features are divided into non-overlapping blocks, and the overlapping blocks are flattened and L2 normalized to obtain a block sequence. A first fully connected network and a random dropout strategy are used to perform cross-channel fusion on the block sequence to obtain the fused representation; The fusion representation is calculated according to the dimension of the block sequence to obtain the global features. The global features are then mapped to labels through a second fully connected network to obtain the classification results, which include the liveness probability and the forgery probability. Based on the classification results, the face liveness detection model is trained using the cross-entropy loss function, the Adam optimizer, and an early stopping strategy. The cross-entropy loss function is: in, Let cross-entropy be the loss function. This serves as the identifier for the training sample sequence. The total number of training samples, Indicates the first The true labels of each training sample For the first The liveness probability of each training sample. For the first The probability of a training sample being faked.

7. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the face liveness detection method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a program, which is executed by a processor to implement the face liveness detection method as described in any one of claims 1-5.

Images