A more fraud-discerning face liveness detection method and its network model

Abstract

The present invention discloses a face liveness detection method and a network model with better fraud discrimination. The method includes constructing an unstructured image input fraud clue generator that weakens the facial structure information of the face, and using multi-scale metric learning embedded in the XBM cross-batch storage mechanism to train the fraud clue generator; inputting the unstructured image and the fraud clue feature map into the auxiliary classifier to carry out supervised classification learning; constructing an overall loss function based on the fraud clue regression loss, triple loss and auxiliary classification loss, and optimizing the network model; in the test phase, constructing the face image to be tested into an unstructured image, inputting it into the fraud clue generator, calculating the fraud score according to the output fraud clue feature map, and determining whether the face image to be tested is a fraud image according to the fraud score. The technical solution of the present application can extract more difficult sample pairs, while weakening invalid facial structure information, thereby improving the accuracy and generalization ability of face liveness detection.

Description

Technical Field

[0001] The present invention relates to fields such as data processing and computer vision, and in particular to a face liveness detection method and a network model thereof that are more fraud-recognizable. Background Art

[0002] Face recognition technology has been widely used in our daily life. However, hackers may deceive face recognition systems through presentation attacks, such as printed photos (i.e., print attacks), digital images / videos (i.e., replay attacks), and 3D masks (i.e., 3D mask attacks). Therefore, in order to ensure the security of face recognition systems, it is necessary to carry out research on face anti-spoofing (FAS) technology.

[0003] Since face anti-spoofing is essentially a binary classification problem, most previous methods use supervised methods to identify live faces and spoofed faces. Traditional face anti-spoofing methods rely on manual descriptors to extract static features, such as LBP, HOG, SIFT, etc., and use traditional classifiers such as SVM for classification. However, manual features are easily affected by environmental conditions, such as camera equipment, lighting conditions, and demonstration attack instruments (PAIs), and traditional methods often do not work very well. With the advancement of hardware and the abundance of data, deep learning networks have made great progress. In these works, deep learning networks are used to extract deep features from pre-trained models and then perform binary classification. The above-mentioned face anti-spoofing (FAS) methods mainly model the decision boundaries of predefined spoofing types. However, the diversity of spoofing samples (including unknown fraud samples) affects the effective decision boundary modeling. In order to mitigate the impact of spoofing diversity on decision boundary modeling, the LGSC (Learning Generalized Spoof Cues for Face) algorithm reconstructs FAS from the perspective of anomaly detection and assumes that live samples belong to a closed set, while different types of spoofing samples are outliers of the closed set and belong to an open set. Based on this assumption, many methods define deception cues as distinguishing features that can be used to distinguish closed sets from open sets. That is, it is assumed that the deception cues of living samples are zero, and different deception samples have different types of deception cues, so as to improve the generalization ability of the model.

[0004] However, directly using the network to extract features does not fully consider the intrinsic connection between samples, that is, it does not propose a real deception model that affects the decision of true and false faces. For example, some deception clues may contain some invalid facial structure information, which interferes with network decisions. On the other hand, the performance of metric learning-based methods depends largely on their ability to mine difficult samples. Although various complex sampling schemes have been developed, such as online difficult sample mining, the algorithm selects the most dissimilar positive samples and the most similar negative samples in a batch to form a triple, and then calculates the loss. However, the ability to mine difficult samples is essentially limited by the batch-size, which makes it difficult to extract enough difficult sample pairs and the network cannot learn good deception clues. This leads to poor generalization ability. Summary of the invention

[0005] The present invention provides a method for detecting live faces with better fraud identification, comprising:

[0006] The face image is sliced ​​and reorganized, and image slices are randomly selected and filled with channel means, while other image blocks remain unchanged, thereby constructing an unstructured image as the input of the network model;

[0007] The unstructured image is input into the fraud clue generator, and the fraud clue feature map is output to obtain the pixel-level loss of fraud clue regression. The fraud clue generator is trained using multi-scale metric learning embedded in the XBM cross-batch storage mechanism to obtain the triplet loss.

[0008] Input the unstructured image and the fraud clue feature map into the auxiliary classifier to obtain the auxiliary classification loss;

[0009] Construct an overall loss function based on fraud clue regression loss, triple loss and auxiliary classification loss, and use the overall loss function to optimize the network model;

[0010] During the testing phase, the face image to be tested is constructed into an unstructured image and input into the optimized fraud clue generator, which outputs a fraud clue feature map. The fraud score is calculated based on the fraud clue feature map, and whether the face image to be tested is a fraud image is determined according to the fraud score.

[0011] As described above, a more fraud-recognizing face liveness detection method is described, wherein an unstructured image is constructed, specifically by decomposing the face image into several blocks, and arranging the face image blocks in any order, and then randomly selecting several image blocks and filling them with channel means, while other image blocks remain unchanged, thereby constructing an unstructured input sample.

[0012] The above-mentioned method for detecting liveness of a face with better fraud identification, wherein constructing an unstructured image specifically comprises the following sub-steps:

[0013] Constructing an unstructured image includes the following sub-steps:

[0014] For the original RGB face image I∈[0,255] 3×H×W , divide it into M×N {I i,j} M×N Grids, these grids have the same height h and width w, where M = H / h, N = W / w;

[0015] After obtaining P = M × N face grids, all the grids are assembled Figure 1 Sample replacement;

[0016] Randomly select P′ (P′<M×N) grids, fill these grids with the pixel average of the channel, and finally form these grids into a new image I'∈[0,255] 3×H×W , the new image I' after unstructured processing is used as the input sample of the network model.

[0017] A more fraud-recognizing face liveness detection method as described above, wherein the fraud clue generator adopts a U-Net architecture to establish jump connections from the encoder to the decoder at multiple scales to generate fraud clues; specifically, ResNet18 pre-trained on ImageNet is selected as the encoder E, which contains 5 encoder residual blocks {E1, E2, E3, E4, E5}; the decoder D is composed of 4 decoder residual blocks {D1, D2, D3, D4}; in each decoder residual block, the feature map of the previous layer is upsampled by nearest neighbor interpolation, and then the features at symmetrical positions in the encoder are added to the decoder through a connection operation to construct feature outputs of different scales; the last residual block of the decoder is connected to a Tanh activation layer to generate fraud clues.

[0018] As described above, a more fraud-discerning face liveness detection method is provided, wherein a multi-scale metric learning technique is used to optimize the model, and specifically includes the following sub-steps:

[0019] Extract multi-scale features from the encoder residual block E5 layer to the decoder residual block D4 layer;

[0020] Use global average pooling to vectorize multi-scale features to obtain a set of feature vectors;

[0021] The triplet loss of the feature vector is calculated to make the distance between the anchor sample and the positive sample as small as possible, and the distance between the anchor sample and the negative sample as large as possible.

[0022] As described above, a more fraud-discriminating face liveness detection method is used, in which a fraud clue generator is trained using multi-scale metric learning embedded in the XBM cross-batch storage mechanism. First, a queue M needs to be created in memory. i , the past feature vectors stored in the queue will be used for mining difficult samples, i∈{E5,D1,D2,D3,D4} represents the encoder E5 and the four decoders {D1,D2,D3,D4}, which are used to store the feature vectors of past iterative training, in represents the feature vector output by the i-th encoder or decoder for the j-th sample, represents the label of the jth sample, q is the queue length, which is usually a multiple of the batch-size;

[0023] The sub-steps of multi-scale metric learning embedded in XBM cross-batch storage mechanism include:

[0024] The network model is pre-trained using the metric learning method to obtain and record the number of iterations k when the feature vector exhibits the "slow drift" phenomenon;

[0025] The early iterative training mines triple samples from a batch of samples, calculates the similarity between samples, and then optimizes the network using triple loss;

[0026] When the number of iterations is greater than k, that is, when the feature vector shows a "slow drift" phenomenon, the model enters the later iterative training; a difficult sample mining strategy based on the XBM cross-batch storage mechanism is adopted. First, triple sample pairs are mined in the queue to obtain the features of a batch of samples, and then the queue is updated, the current batch features are entered into the queue, and the earliest batch features are removed from the queue. Then, the similarity between samples is calculated, and finally, the network is optimized using the triplet loss.

[0027] As described above, a more fraud-recognizing face liveness detection method is proposed, wherein the overall loss function is:

[0028]

[0029] where i∈{E5,D1,D2,D3,D4} represents the layer to which the triplet loss penalty is applied, and α 1 , α 2 , α 3 is a hyperparameter used to balance the importance of different loss components; L r Pixel-level loss for fraud clue regression; is the triple loss; is the triple loss based on XBM cross-batch storage mechanism; L a is the auxiliary classification loss.

[0030] As described above, a more fraud-discriminating face liveness detection method is used, in which, in the test phase, the generated fraud clue map is used instead of the output of the classifier to perform face liveness detection, and the test sample is fed into the trained network model to obtain the fraud clue map of the test sample. Statistical fraud clues grayscale average As the fraud score of the test sample, the higher the fraud score, the greater the possibility that the test sample is a fraudulent image.

[0031] The present application also provides a network model, characterized in that it includes: a fraud clue generator and an auxiliary classifier; and uses any of the above-mentioned face liveness detection methods with higher fraud discrimination to perform face liveness detection.

[0032] The beneficial effects achieved by the present invention are as follows: The present invention first performs unstructured processing on the input samples to destroy the facial structure information of the human face and weaken the influence of noise that is irrelevant to the decision-making of the boundary between the live body and the prosthesis. The U-Net architecture is used as a fraud feature generator to generate multi-scale fraud clues, and only the regression constraints of the live samples are added to the objective function to ensure the diversity of the fraud clues. In the network, multi-scale metric learning technology is introduced, and the cross-batch storage mechanism is used to extract more difficult sample pairs to improve the intra-class similarity and inter-class discreteness. Finally, an auxiliary classifier is used as an amplifier of fraud clues to obtain more discriminative fraud clues. The technical solution of the present application can extract more difficult sample pairs and weaken invalid facial structure information, thereby improving the accuracy and generalization ability of face anti-spoofing (FAS) detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0034] Figure 1 This is a flow chart of a face liveness detection method with better fraud identification provided in Example 1 of the present application;

[0035] Figure 2 It is a schematic diagram of the overall network structure of the network model provided by this application;

[0036] Figure 3 It is a schematic diagram of the feature vector and offset that change with the number of iterations;

[0037] Figure 4 This is a schematic diagram of XBM’s cross-batch storage mechanism. DETAILED DESCRIPTION

[0038] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0039] Embodiment 1

[0040] like Figure 1 As shown, the first embodiment of the present application provides a face liveness detection method with better fraud identification, including:

[0041] Step 110: Slice and reorganize the face image, and randomly select image slices and fill them with channel means, while other image blocks remain unchanged, thereby constructing an unstructured image as a network model input;

[0042] Since the face image contains various feature information, including individual facial features, distinguishing features between live and prosthesis, etc., and the distinguishing features of individual faces are generally more active, while the distinguishing features between live and prosthesis are more subtle, active individual facial features can easily affect the results of face liveness detection. Since the fraud texture in the face image has the global shift-invariant property, the present application will not affect the extraction of fraud features by slicing and reorganizing the face image, but it can destroy the facial structure information of the face, which can weaken the influence of noise irrelevant to the judgment of liveness and prosthesis.

[0043] Specifically, after the face image is collected, the data preprocessing stage is first entered, in which the face image is decomposed into several blocks, and the face image blocks are arranged in any order. Then, several image blocks are randomly selected and filled with the channel mean, while other image blocks remain unchanged, thereby constructing an unstructured input sample.

[0044] Among them, the unstructured image processing specifically includes segmentation, splicing and random filling, specifically:

[0045] ①For the original RGB face image I∈[0,255] 3×H×W , divide it into M×N {I i,j} M×N Grids, these grids have the same height h and width w, where M = H / h, N = W / w;

[0046] ② After obtaining P = M × N face grids, all the grids are assembled Figure 1 Sample replacement;

[0047] ③ Randomly select P′ (P′<M×N) grids, fill these grids with the pixel average of the channel, and finally form these grids into a new image I'∈[0,255] 3×H×W , the new image I' after unstructured processing is used as the input sample of the network model.

[0048] This application uses unstructured image reconstruction to destroy the facial structure of the human face, reduce the noise in the input sample that is irrelevant to face liveness detection, which is beneficial to face liveness detection. In addition, the original image is divided into P grids, and P! different arrangements can be obtained, implicitly achieving the effect of data expansion.

[0049] Step 120: input the unstructured image into the fraud clue generator, output the fraud clue feature map, obtain the pixel-level loss of the fraud clue regression, and use the multi-scale metric learning embedded in the XBM cross-batch storage mechanism to train the fraud clue generator to obtain the triple loss;

[0050] Figure 2 The overall network structure diagram of the network model provided in this application. The network model is composed of a fraud clue generator and an auxiliary classifier. For the fraud clue generator, a supervised method is used to learn fraud clues to minimize the fraud clues of living samples; the auxiliary classifier is used as an amplifier of fraud clues to help learn more discriminative fraud clues.

[0051] In the embodiment of the present application, the fraud clue generator adopts the U-Net architecture to establish jump connections from the encoder to the decoder at multiple scales to generate fraud clues. Specifically, ResNet18 pre-trained on ImageNet is selected as the encoder E, which contains 5 encoder residual blocks {E1, E2, E3, E4, E5}; the decoder D consists of 4 decoder residual blocks {D1, D2, D3, D4}; in each decoder residual block, the feature map of the previous layer is upsampled by nearest neighbor interpolation, and then the features of the symmetrical position in the encoder are added to the decoder through the connection operation to construct feature outputs of different scales; the last residual block of the decoder is connected to a Tanh activation layer to generate fraud clues.

[0052] Since the live sample does not have any fraudulent material composition, the fraud clues of the live sample should be zero. Fraud samples of different materials should have different fraud clues, and fraud clues only exist in fraud samples. Given the input RGB image I, the fraud clue generator outputs a fraud clue map C of the same size. The fraud clue map C of the live sample is a feature map of all zeros; while C of the fraud sample is unknown. The pixel-level loss expression of fraud clue regression is:

[0053]

[0054] In formula (1), L r represents the pixel-level loss for fraud clue regression; N l is the number of living samples; I i is the i-th unstructured living sample input; C i is the corresponding fraud clue mapping diagram; this objective function only constrains the training of living samples, minimizing the fraud clues of living samples, but does not constrain fraud samples, thus ensuring the diversity of fraud forms.

[0055] In order to improve the intra-class compactness and inter-class separability in face liveness detection, this application uses multi-scale metric learning technology to optimize the model, which specifically includes the following sub-steps:

[0056] ① Extract multi-scale features from the encoder residual block E5 layer to the decoder residual block D4 layer;

[0057] ②Use global average pooling to vectorize multi-scale features to obtain a set of feature vectors;

[0058] ③ Calculate the triplet loss of the feature vector to minimize the distance between the anchor sample and the positive sample, and maximize the distance between the anchor sample and the negative sample; the expression of the triplet loss function is:

[0059]

[0060] In formula (2), T is the number of sample pairs, a i represents the sample, p i represents the positive sample, n i represents a negative sample, m is a predefined boundary constant, and d(i,j) represents the Euclidean distance between two normalized feature vectors, which is calculated as follows:

[0061]

[0062] In formula (3), v i and v j represents two different samples in the triplet, d(a i ,p i ) represents the Euclidean distance between the scanned sample and the positive sample; d(a i ,n i ) represents the Euclidean distance between the scanned sample and the negative sample.

[0063] In the embodiment of the present application, there is a "slow drift" phenomenon in the update of the feature vector during metric learning, that is, the feature differences calculated for the same sample in different training iterations are small. In order to quantify the "slow drift", when the step size is set to Δt, the feature drift of the input sample x is defined as:

[0064]

[0065] In formula (4), D(x, t; Δt) is the feature offset after Δt time, f(x; θ t ) is the feature extracted from sample x at time t, θ is the model parameter; f(x; θ t-Δt ) is the feature extracted from sample x at time t-Δt. When the feature offset stabilizes at a small value, it means that a “slow drift” phenomenon has occurred.

[0066] Formula (4) can be used to calculate the offset of the feature vector of the same sample as the number of iterations increases. Figure 3 As shown in the figure, in the early stage of iteration, the features change greatly and the feature drift is still obvious; but after about 2500 iterations, the features become relatively stable and the feature offset is small, especially when the learning rate decreases, the feature offset is even smaller, and the feature vector shows a "slow drift" phenomenon.

[0067] When the "slow drift" phenomenon occurs in the feature vector, it means that there is only a small difference between the feature vector trained in the past and the current feature vector. The features of the past training can be temporarily stored and used for the current difficult sample mining, which can expand the mining scope of difficult samples without any additional calculations, but only requires a small amount of additional storage space. Based on this, the present application embeds XBM (cross-batch memory) into the metric learning module. The cross-batch memory mechanism can be used to store the feature vectors of past iterations, allowing the model to collect enough difficult sample pairs on multiple batches of data or even the entire data set. XBM can be integrated into a deep learning framework based on metric learning as a plug-and-play module to improve the performance of metric learning. The XBM method is not available in the early stage of model training, but can only be used when the "slow drift" phenomenon occurs in the feature vector.

[0068] In order to use the XBM method, you need to create some queues M in memory. i , the past feature vectors stored in the queue will be used for mining difficult samples, i∈{E5,D1,D2,D3,D4} represents the encoder E5 and the four decoders {D1,D2,D3,D4} respectively. The queue is used to store the feature vectors of past iterative training. in represents the feature vector output by the i-th encoder or decoder for the j-th sample, represents the label of the jth sample, and q is the queue length, which is usually a multiple of the batch-size. Figure 4As shown in Figure 1, the main work of the XBM module is to maintain and update the storage queue. In each iteration, triple sample pairs are mined from the queue to obtain sample features, and then the feature vectors and labels of the current batch are added to the queue, and the feature vectors and labels of the earliest batch are removed from the queue, always keeping the queue length q.

[0069] In the embodiment of the present application, a fraud clue generator is trained using multi-scale metric learning embedded in the XBM cross-batch storage mechanism, specifically including the following sub-steps:

[0070] ① Use the metric learning method to pre-train the network model, obtain and record the number of iterations k when the feature vector exhibits the "slow drift" phenomenon, and then formally enter the model training process.

[0071] The entire model training process is divided into two stages: early iterative training and late iterative training.

[0072] ② The early iterative training adopts a batch-all difficult sample mining strategy to obtain triple samples in a batch of sample blocks, calculate the similarity between samples, and then use the triplet loss to optimize the network.

[0073] ③ When the number of iterations is greater than k, that is, the feature vector shows a "slow drift" phenomenon, the model enters the later iterative training. At this time, the XBM difficult sample mining strategy is adopted, that is, firstly, the triple sample pairs are mined in the queue to obtain the sample features, and then the queue is updated, the current batch features are put into the queue, and the earliest batch features are removed from the queue, and then the similarity between the samples is calculated, and finally the network is optimized using the triple loss;

[0074] In the early iteration, the model uses formula (3) as the objective function to optimize the network; in the later iteration, when using the XBM difficult sample mining strategy, the following objective function is used to optimize the network:

[0075]

[0076] In the formula, p i_xbm and n i_xbm These are valid positive and negative samples mined from the entire storage queue.

[0077] Step 130: input the unstructured image and the fraud clue feature map into the auxiliary classifier to obtain the auxiliary classification loss;

[0078] After the fraud clues are generated, the generated fraud clue map C is added to the input image I to form a superimposed image S, which is used as the input of the auxiliary classifier. The auxiliary classification loss is the binary cross entropy loss:

[0079]

[0080] In formula (6), N is the number of samples, z i is the sample label, q i is the classification probability.

[0081] Step 140: construct an overall loss function based on the fraud clue regression loss, the triplet loss, and the auxiliary classification loss, and use the overall loss function to optimize the network model;

[0082] The loss function of the network model of this application is divided into three parts: the pixel-level loss Lr of fraud clue regression of live samples, the triple loss Lt or Lt_xbm on live samples and fraud samples, and the auxiliary binary classification loss La. Combining these losses, the total loss of model training is obtained:

[0083]

[0084] In formula (7), i∈{E5,D1,D2,D3,D4} represents the layer to which the triplet loss penalty is applied, α 1 , α 2 , α 3 is a hyperparameter used to balance the weights of different loss components.

[0085] Step 150: In the test phase, the face image to be tested is constructed into an unstructured image, which is input into the optimized fraud clue generator, a fraud clue feature map is output, a fraud score is calculated according to the fraud clue feature map, and whether the face image to be tested is a fraud image is determined according to the fraud score;

[0086] In the testing phase, this application uses the generated fraud clue map instead of the output of the classifier to perform face liveness detection, and sends the test sample to the trained network to obtain the fraud clue map of the test sample. Statistical fraud clues grayscale average As the fraud score of the test sample, the higher the fraud score, the greater the possibility that the test sample is a fraudulent image.

[0087] To verify the performance of the algorithm of the present invention, the following performance indicators are used:

[0088] (1) Attack Presentation Classification Error Rate (APCER). APCER is the maximum value of FAR (False Accept Rate) of various types of prosthetic attacks. FAR is the ratio of prosthetic faces to living faces.

[0089] (2) Bona Fide Presentation Classification Error Rate (BPCER): BPCER is the rate at which live faces are mistakenly classified as fake faces.

[0090] (3) Average Classification Error Rate (ACER). To calculate ACER, we first need to find the threshold on the validation set that makes APCER equal to BPCER. Then, we calculate APCER and BPCER at the specified threshold on the test set. The average of APCER and BPCER is ACER.

[0091] The algorithm of the present invention was internally tested on four protocols of the OULU-NPU dataset and compared with the latest related algorithms (MILHP, STASN, FaceDs, LGSC, Auxiliary and GRADIENT). The results are shown in Table 1:

[0092] Table 1. Internal testing on 4 protocols on the OULU-NPU dataset and comparison with state-of-the-art methods

[0093]

[0094] From the data in the table, we can see that the APCER indicators of the algorithm of the present invention in the face liveness detection experiment on the four protocols are 0.8, 0.7, 1.3±1.2, and 3.2, respectively, which are better than other comparison algorithms; in the experiments of protocols 2 and 3, the BPCER of the algorithm is slightly inferior to the STASN algorithm; in the experiment of protocol 4, the BPCER of the algorithm is slightly inferior to the LGSC algorithm; in the experiments of the four protocols, the average classification error rate ACER is 0.4, 0.6, 1.2±1.2, and 3.2, respectively, which are better than other comparison algorithms. It can be seen that the algorithm of the present invention is generally ahead of the latest methods in terms of classification accuracy and generalization ability.

[0095] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting live faces with better fraud identification, characterized in that: include: The face image is sliced ​​and reorganized, and image slices are randomly selected and filled with channel means, while other image blocks remain unchanged, thereby constructing an unstructured image as the input of the network model; The unstructured image is input into the fraud clue generator, and the fraud clue feature map is output to obtain the pixel-level loss of fraud clue regression. The fraud clue generator is trained using multi-scale metric learning embedded in the XBM cross-batch storage mechanism to obtain the triplet loss. Input the unstructured image and the fraud clue feature map into the auxiliary classifier to obtain the auxiliary classification loss; Construct an overall loss function based on fraud clue regression loss, triple loss and auxiliary classification loss, and use the overall loss function to optimize the network model; The overall loss function is: Where i∈{E5,D1,D2,D3,D4} represents the layer to which the triplet loss penalty is applied, α1, α2, α3 are hyperparameters used to balance the importance of different loss components; L r Pixel-level loss for fraud clue regression; is the triple loss; is the triple loss based on XBM cross-batch storage mechanism; L a is the auxiliary classification loss; In the testing phase, the face image to be tested is constructed into an unstructured image, which is input into the optimized fraud clue generator, and a fraud clue feature map is output. The fraud score is calculated based on the fraud clue feature map, and whether the face image to be tested is a fraud image is determined according to the fraud score; To train the fraud clue generator using multi-scale metric learning embedded in the XBM cross-batch storage mechanism, we first need to create a queue M in memory. i , the past feature vectors stored in the queue will be used for mining difficult samples, i∈{E5,D1,D2,D3,D4} represents the encoder E5 and the four decoders {D1,D2,D3,D4}, which are used to store the feature vectors of past iterative training, in represents the feature vector output by the i-th encoder or decoder for the j-th sample, represents the label of the jth sample, and q is the queue length; The sub-steps of multi-scale metric learning embedded in XBM cross-batch storage mechanism include: The network model is pre-trained using the metric learning method to obtain and record the number of iterations k when the feature vector exhibits the "slow drift" phenomenon; The early iterative training mines triple samples from a batch of samples, calculates the similarity between samples, and then optimizes the network using triple loss; When the number of iterations is greater than k, that is, the feature vector shows "slow drift", the model enters the later iterative training; a difficult sample mining strategy based on the XBM cross-batch storage mechanism is adopted. First, triple sample pairs are mined in the queue to obtain the features of a batch of samples, and then the queue is updated, the current batch features are entered into the queue, and the earliest batch features are removed from the queue. Then, the similarity between samples is calculated, and finally, the network is optimized using triplet loss.

2. The method for face liveness detection with higher fraud identification as claimed in claim 1, characterized in that: Constructing an unstructured image, specifically: decomposing the face image into several blocks, and arranging the face image blocks in any order, and then randomly selecting several image blocks and filling them with channel means, while other image blocks remain unchanged, thereby constructing an unstructured input sample.

3. The method for face liveness detection with higher fraud identification as claimed in claim 2, characterized in that: Constructing an unstructured image includes the following sub-steps: For the original RGB face image I∈[0,255] 3×H×W , divide it into M×N {I i,j } M×N Grids, these grids have the same height h and width w, where M = H / h, N = W / w; After obtaining P = M × N face grids, all grids are replaced like puzzle pieces; Randomly select P′ (P′<M×N) grids, fill these grids with the pixel average of the channel, and finally form these grids into a new image I′∈[0,255] 3×H×W , the new image I′ after unstructured processing is used as the input sample of the network model.

4. The method for face liveness detection with higher fraud identification as claimed in claim 1, characterized in that: The fraud clue generator adopts the U-Net architecture and establishes jump connections from the encoder to the decoder at multiple scales to generate fraud clues. Specifically, it selects ResNet18 pre-trained on ImageNet as the encoder E, which contains 5 encoder residual blocks {E1, E2, E3, E4, E5}; the decoder D consists of 4 decoder residual blocks {D1, D2, D3, D4}; in each decoder residual block, the feature map of the previous layer is upsampled by nearest neighbor interpolation, and then the features at symmetrical positions in the encoder are added to the decoder through a connection operation to construct feature outputs of different scales; the last residual block of the decoder is connected to a Tanh activation layer to generate fraud clues.

5. The method for face liveness detection with higher fraud identification as claimed in claim 4, characterized in that: The model is optimized using multi-scale metric learning technology, which includes the following sub-steps: Extract multi-scale features from the encoder residual block E5 layer to the decoder residual block D4 layer; Use global average pooling to vectorize multi-scale features to obtain a set of feature vectors; The triplet loss of the feature vector is calculated to make the distance between the anchor sample and the positive sample as small as possible, and the distance between the anchor sample and the negative sample as large as possible.

6. The method for detecting live faces with better fraud identification as claimed in claim 1, characterized in that: In the testing phase, the generated fraud clue map is used instead of the output of the classifier for face liveness detection. The test sample is fed into the trained network model to obtain the fraud clue map of the test sample. Statistical fraud clues grayscale average As the fraud score of the test sample, the higher the fraud score, the greater the possibility that the test sample is a fraudulent image.

Images