Robust face liveness detection method and system based on adaptive graph neural network

Abstract

The present invention belongs to the field of face recognition technology, and discloses a robust face liveness detection method and system based on an adaptive graph neural network. The method divides face image training samples into support set samples and query set samples according to a set ratio, and performs data preprocessing and data enhancement processing on the support set samples and the query set samples in batches. Finally, the face image is divided into image blocks of the same size through block processing; the image blocks are vectorized, the face image features are extracted, the spatial position information of the image blocks is introduced, each divided image block is regarded as a node, and a dynamic K nearest neighbor dense algorithm is used to adaptively construct a topology graph, and the update of the loop edge and the update process of the node are obtained to obtain the optimal topology graph; the classifier determines whether it is a live or disguised face according to the feature information of the optimal topology graph. The present invention improves the accuracy, adaptability and credibility of face liveness detection in the face of unknown attacks.

Description

Technical Field

[0001] The present invention belongs to the technical field of face recognition, and in particular relates to a robust face liveness detection method and system based on an adaptive graph neural network. Background Art

[0002] With the widespread application of face recognition technology, face disguise is becoming more frequent. Common disguise attacks include printing photos, screen replay attacks, 3D masks, etc. These disguise attacks have security issues such as infringing personal privacy and reputation rights and spreading false information. They pose a serious threat to the financial, security, medical and payment fields that use face recognition technology. Face anti-disguise, also known as face liveness detection (FAS), is a self-evolving technology, that is, the iterative development of attack and defense. With the development of technologies such as 3D and artificial intelligence, the samples generated by disguise attacks are becoming more realistic, and the gap between disguised samples and live samples is becoming more subtle, which increases the difficulty of disguise detection; with the increase in face disguise application scenarios, disguise attack forms are constantly being renewed, and unknown disguises are emerging in an endless stream, making the disguise detection task more severe. Obviously, the continuous advancement of disguise attack technology has put forward higher requirements for the accuracy, adaptability, and generalization of liveness detection.

[0003] Face liveness detection methods mainly distinguish between real and fake faces by analyzing the difference features between live faces and disguised faces. Its essence is a binary classification problem, which aims to detect whether the input sample is a live face or a disguised face. Early face liveness detection methods were based on traditional biometrics for detection, and usually relied on analyzing static features of the face, such as texture, color, and shape, to distinguish between real faces and disguised faces. The disadvantage of this method is that it is easily affected by changes in lighting, occlusion, and posture, and it is difficult to identify advanced disguise methods, such as synthetic videos and 3D masks, making the face liveness detection method based on traditional biometrics not ideal. With the advancement of deep learning technology and the popularity of large-scale annotated datasets, researchers have developed a variety of binary classification deep disguise detection (FAS) network architectures to distinguish between real and disguised faces. However, factors such as the bias and limitations of the dataset, the complexity of the model, the lack of effective domain adaptation and multimodal data fusion, etc., have affected the generalization ability of these methods, and may cause overfitting problems under specific sample attacks.

[0004] Compared with traditional convolutional neural networks, graph neural networks are more suitable for processing unstructured data, and they have powerful complex relationship expression capabilities. In the field of computer vision, data such as images or videos are regular grid data, which are usually processed using traditional convolutional neural networks. However, the effective semantic information in the image is irregular, and there may be complex spatial relationships. The semantic information may have a hierarchical structure. Therefore, the inductive bias of convolutional neural networks is likely to limit their performance in image processing, while graph neural networks usually have weaker inductive bias, which makes graph neural networks have greater application potential in the field of computer vision. Summary of the invention

[0005] The present invention proposes a robust face liveness detection method and system based on an adaptive graph neural network, which improves the generalization ability by introducing a graph neural network, utilizes its advantages of global feature extraction and unstructured data representation, enhances the interpretability and visualization effect of the robust face liveness detection model in the face liveness detection task, adopts a dynamic K nearest neighbor dense algorithm to adaptively determine the number of neighbors K, and uses an adaptive graph convolution module to dynamically update the topology map to obtain the optimal topology map. At the same time, a meta-learning strategy is introduced in the training process of the robust face liveness detection model with an improved combined soft label target loss function to solve the problem that the graph neural network and face liveness detection are easily limited by the insufficient number of samples in practical applications, so that the robust face liveness detection model can quickly adjust the parameters of the robust face liveness detection model and improve the overall performance when facing a new task with a small number of unknown attack samples, thereby improving the robustness, generalization ability, rapid adaptability and credibility of the robust face liveness detection model.

[0006] The present invention is implemented by the following technical scheme: a robust face liveness detection method based on an adaptive graph neural network, wherein the adopted robust face liveness detection model includes a data processing module, a feature extraction module, a topology map construction and update module and a classifier according to functional division, and the steps are as follows:

[0007] S1: The data processing module divides the face image training samples into support set samples and query set samples according to the set ratio, and performs data preprocessing and data enhancement processing on the support set samples and query set samples in batches. Finally, the face image is divided into image blocks of the same size through block processing;

[0008] S2: vectorize the image blocks through the feature extraction module to extract the facial image features;

[0009] S3: The topology map construction and update module includes a dynamic K-nearest neighbor dense algorithm and an adaptive graph convolution module. According to the features of the face image, the spatial position information of the image block is introduced, and each divided image block is regarded as a node. The dynamic K-nearest neighbor dense algorithm is used to adaptively construct the topology map. The update of the topology map includes the update of the edge and the update of the node. The node feature is updated through the adaptive graph convolution module (AGCN). After the node feature is updated, the edge is updated through the dynamic K-nearest neighbor dense algorithm. The edge update and node update process are cyclically performed to obtain the optimal topology map.

[0010] S4: The classifier determines whether it is a living or disguised face based on the feature information of the optimal topological map.

[0011] Further preferably, in step S1, the data preprocessing is to randomly select flipping, color dithering, erasing and mixing images to process the sample data.

[0012] Further preferably, in step S1, the data enhancement processing is to randomly select a mixed data enhancement (Mixup) or a cut mixed data enhancement (Cutmix) method to perform data enhancement.

[0013] Further preferably, the method of adaptively constructing the topological graph using the dynamic K nearest neighbor dense algorithm is as follows: each image block is converted into a feature vector and used as a node, and the node feature Includes node embedding features With position encoding , get an unordered set of nodes ; represents the i-th node, i∈{1,2,…,N}, N is the number of nodes. For each node, the pairwise distance between nodes is calculated by the defined metric function, and then the K nearest neighbor nodes are selected by the dynamic K nearest neighbor dense algorithm. The neighbor node set is expressed as , and for all neighboring nodes , construct connection nodes and nodes Edge , and then get the graph , Represents a set of edges.

[0014] Further preferably, the processing process of the dynamic K nearest neighbor dense algorithm is as follows:

[0015] First, based on the node characteristics Calculate the pairwise distances between nodes and the average distance from each node to all other nodes , and then calculate the inverse density of each node , and normalize the inverse density;

[0016] (1);

[0017] in, represents the normalized inverse density of each node, is the maximum normalized inverse density, is the minimum normalized inverse density;

[0018] Take the median to determine the K value of the sample, and first calculate the median benchmark: , and then calculate the K value based on the median basis:

[0019] (2);

[0020] Among them, quantile(•) is a statistical function, Indicates rounding to the nearest integer. is the maximum K value set, is the minimum K value set;

[0021] Finally, boundary conditions are applied to ensure that the K value is within the set arrive between:

[0022] (3).

[0023] Further preferably, in step S3, updating of nodes is to achieve information exchange between nodes by aggregating and updating embedded features of neighboring nodes:

[0024] (4);

[0025] in, is the updated i-th node, and are the learnable parameters for aggregation and update operations, respectively. and They are aggregation and update functions respectively; the aggregation operation calculates the representation of the node by aggregating the embedded features of neighboring nodes, and the update operation further performs nonlinear transformation on the aggregated embedded features;

[0026] The adaptive graph convolution module uses maximum correlation graph convolution (MRGCN) to capture embedded features. The aggregation and update formulas are as follows:

[0027] (5) ;

[0028] (6);

[0029] For Node Its neighbor nodes The embedded features obtained by aggregation through maximum correlation graph convolution, is The embedded features obtained after updating;

[0030] The whole process is expressed as: , is the updated embedding feature, and AGCN represents the operation of the adaptive graph convolution module.

[0031] Further preferably, the adaptive graph convolution module consists of an adaptive graph convolution block and a downsampling layer, using a total of twelve adaptive graph convolution blocks, divided into four parts of 2, 2, 6, and 2. Each part is connected through a downsampling layer, and the feature dimension of each part is constantly increasing; the adaptive graph convolution block integrates a graph convolution network (Grapher) and a feedforward neural network (FFN), and both the graph convolution network (Grapher) and the feedforward neural network (FFN) use residual connections.

[0032] Further preferably, the graph convolution network (Grapher) is composed of a first two-dimensional convolution (Conv2D), a first fully connected layer, a second two-dimensional convolution, a second fully connected layer, a parameterized rectified linear unit activation function (PreLu), a maximum correlation graph convolution block (MRGCN), a third two-dimensional convolution (Conv2D), a third fully connected layer and a first path discarding block (Droppath) in sequence.

[0033] Further preferably, the feedforward neural network part (FFN) is composed of a fourth two-dimensional convolution (Conv2D), a fourth fully connected layer, a parameterized rectified linear unit activation function (PreLu), a fifth two-dimensional convolution (Conv2D), a fifth fully connected layer and a second path discarding block (Droppath) in sequence.

[0034] Further optimization, in the training stage, the training stage of the robust face liveness detection model is transformed into two stages: inner loop training and outer loop training. The meta-learning parameters are updated and optimized in the inner loop training stage; in the outer loop training stage, the meta-learning parameters adapted by the inner loop training are first loaded, and then the overall parameters of the robust face liveness detection model are updated, and the classifier parameters are selected as the meta-learning parameters. .

[0035] Further preferably, the objective function of the inner loop training selects KL divergence loss, and the objective function of the outer loop training selects soft label cross entropy loss.

[0036] The present invention also provides a robust face liveness detection system based on an adaptive graph neural network, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the robust face liveness detection method based on an adaptive graph neural network of the present invention is implemented.

[0037] The present invention has the following advantages:

[0038] 1. The adaptive graph convolution module adopts a pyramid structure design, which enables the robust face liveness detection model to analyze and fuse features at multiple scales, and comprehensively capture facial feature information from local to global.

[0039] 2. The dynamic K-nearest-neighbor dense algorithm is used to achieve adaptive adjustment of the topological map, which can dynamically update and determine the set of neighbor nodes according to different node densities, so that the robust face liveness detection model can extract and refine features at different scales, thereby more accurately capturing facial features and enhancing the recognition ability of complex disguised faces.

[0040] 3. Introducing a meta-learning strategy during the training of the robust face liveness detection model enables the robust face liveness detection model to exhibit rapid adaptability when facing new attack scenarios or a small number of training samples. The inner-loop and outer-loop separation training process adopted by the meta-learning strategy can avoid the problem of conflicts that may arise when rapid adaptation and fine-tuning are performed simultaneously, enabling the robust face liveness detection model to quickly adjust and continuously optimize when facing new tasks, thereby improving the generalization ability and robustness of the robust face liveness detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is the structural diagram of the robust face liveness detection model;

[0042] Figure 2 Schematic diagram of the adaptive graph convolution module structure;

[0043] Figure 3 Schematic diagram of the adaptive graph convolution block structure;

[0044] Figure 4 Optimizing flow chart for meta-learning strategies. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0046] It should be understood that when the terms "first", "second", etc. are used in the claims, descriptions, and drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the description and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0047] like Figure 1 As shown in the figure, the robust face liveness detection method based on adaptive graph neural network adopts a robust face liveness detection model which includes a data processing module, a feature extraction module, a topology map construction and update module and a classifier according to its functions. The steps are as follows:

[0048] S1: The data processing module divides the face image training samples into support set samples and query set samples according to the set ratio, and performs data preprocessing and data enhancement processing on the support set samples and query set samples in batches. Finally, the face image is divided into image blocks of the same size through block processing;

[0049] S2: vectorize the image blocks through the feature extraction module to extract the facial image features;

[0050] S3: The topology map construction and update module includes a dynamic K-nearest neighbor dense algorithm and an adaptive graph convolution module. According to the features of the face image, the spatial position information of the image block is introduced, and each divided image block is regarded as a node. The dynamic K-nearest neighbor dense algorithm is used to adaptively construct the topology map. The update of the topology map includes the update of the edge and the update of the node. The node feature is updated through the adaptive graph convolution module (AGCN). After the node feature is updated, the edge is updated through the dynamic K-nearest neighbor dense algorithm. The edge update and node update process are cyclically performed to obtain the optimal topology map.

[0051] S4: The classifier determines whether it is a living or disguised face based on the feature information of the optimal topological map.

[0052] In step S1, before training, the face image training samples are firstly divided into Randomly divide into non-overlapping support set samples and query set sample ,In order to improve data quality and enhance the generalization ability of the ,robust face liveness detection model, the sample data is preprocessed by ,flipping, color jittering, erasing and mixing images randomly ,selecting the mixed data enhancement (Mixup) or cutting mixed data enhancement (Cutmix) method for ,data enhancement, and the soft label value of the mixed image is ,obtained according to the ratio of the mixed image.

[0053] The Mixup method is to generate an intermediate sample by mixing two samples in a certain ratio. For example, With the bth picture According to the following formula: Mix them together to produce a mixed image ,in, is a randomly generated mixing factor, and the corresponding soft label is according to the formula: Produced, among which, are the effective coded hard labels of the a-th image and the b-th image respectively. are the soft labels of the generated hybrid images.

[0054] The Cutmix method is similar to the Mixup method, except that the Cutmix method generates new samples by splicing image blocks. For example, the bth image in a batch is of Partially spliced ​​to the ath picture Then, the soft labels of the spliced ​​images are generated in the same way as Mixup.

[0055] Graph convolutional neural networks only accept data with graph structures as input, so face image samples need to be converted into a topological graph consisting of nodes and edges.

[0056] When constructing a topological graph, the present invention not only considers the grayscale embedding feature information between nodes, but also considers the spatial position information between different nodes, so that the spatial information of the data can be fully utilized and spatially adjacent nodes can be preferentially selected as neighbors. Includes node embedding features With position encoding : , where the embedding features of the nodes are is a learnable feature, and the position encoding It is a fixed relative position code, that is, by quantizing the coordinate differences of each pixel (node) in the sequence, the relative position information between them is captured, and these differences are encoded through a mathematical model. When constructing a topological map, if the number of neighbors K is set too large, the amount of aggregation calculations in the adaptive graph convolution module will increase the computational burden of the robust face liveness detection model; if the number of neighbors K is set too small, more detailed information may be discarded. The use of a fixed number of neighbors K in the traditional K nearest neighbor algorithm is obviously unable to adapt to data changes, which may lead to a decrease in model performance or a decrease in computational efficiency. To this end, the present invention designs an algorithm that dynamically determines the K value based on the local density of the data, called the dynamic K nearest neighbor dense algorithm, and the process is as follows:

[0057] First, based on the node characteristics Calculate the pairwise distances between nodes and calculate the average distance from each node to all other nodes (within the same image, excluding its own nodes) , and then calculate the inverse density of each node , and normalize the inverse density according to formula (1):

[0058] (1);

[0059] in, represents the normalized inverse density of each node, is the maximum normalized inverse density, is the minimum normalized inverse density.

[0060] Considering the risk of the sample density being too large or too small, the K value of the sample is determined by taking the median. First, the median benchmark is calculated: , and then calculate the K value based on the median benchmark. The calculation formula is shown in formula (2):

[0061] (2);

[0062] Here, quantile(•) is a statistical function used to calculate the quantile in a given data set. Indicates rounding to the nearest integer. is the maximum K value set, is the minimum K value set.

[0063] Finally, boundary conditions are applied to ensure that the K value is within the set arrive between:

[0064] (3).

[0065] In summary, the composition process of the present invention is: each image block is converted into a feature vector and used as a node, and the node feature Includes node embedding features With position encoding , get an unordered set of nodes ; represents the i-th node, i∈{1,2,…,N}, N is the number of nodes. For each node, the pairwise distance between nodes is calculated by the defined metric function, and then the K nearest neighbor nodes are selected by the dynamic K nearest neighbor dense algorithm. The neighbor node set is expressed as , and for all neighboring nodes , construct connection nodes and nodes Edge , then we can get the graph , Represents a set of edges.

[0066] In step S3 of this embodiment, the updating of nodes is to realize information exchange between nodes by aggregating and updating the embedded features of neighboring nodes. The general node updating formula is:

[0067] (4);

[0068] in, is the updated i-th node, and are the learnable parameters for aggregation and update operations, respectively. and They are aggregation and update functions respectively; the aggregation operation calculates the representation of the node by aggregating the embedded features of neighboring nodes, and the update operation further performs nonlinear transformation on the aggregated embedded features.

[0069] The adaptive graph convolution module uses maximum correlation graph convolution (MRGCN) to capture subtle embedding features. The aggregation and update formulas are as follows:

[0070] (5) ;

[0071] (6);

[0072] For Node Its neighbor nodes The embedded features obtained by aggregation through maximum correlation graph convolution, is The embedded features obtained after updating;

[0073] The whole process is expressed as: , is the updated embedding feature, and AGCN represents the operation of the adaptive graph convolution module.

[0074] In summary, the dynamic update process of each round of topology graph is: first, the embedded features of the nodes are Perform aggregation and update operations to obtain updated embedding features: Then, position encoding is introduced into the updated embedding feature Get the updated node features: . Then according to the updated node characteristics The pairwise distances between nodes are calculated, and the top K nearest neighbors of each node are returned through the dynamic K nearest neighbor dense algorithm, thereby updating the topology map. After that, the next round of topology map update will begin. Each round of topology map update will cycle between node update and edge update. Through the iterative conduction of gradients, global semantic features can be gathered.

[0075] like Figure 2 As shown, the adaptive graph convolution module of this embodiment adopts a pyramid-type module structure. Compared with the traditional isotropic structure, this design can reduce the number of parameters while maintaining accuracy, making the adaptive graph convolution module more robust when processing image-based datasets, and can effectively capture the multi-scale features of face samples with lower computational complexity. Figure 2 In the above equation, H is the length of the image block, W is the width of the image block, and D is the feature dimension. is the K value determined by the dynamic K nearest neighbor dense algorithm. The adaptive graph convolution module consists of an adaptive graph convolution block and a downsampling layer. A total of twelve adaptive graph convolution blocks are used, which are divided into four parts: 2, 2, 6, and 2. Each part is connected through a downsampling layer, and the feature dimension of each part is continuously increasing, which is {80, 160, 400, 640} respectively. This can optimize the depth and width of the adaptive graph convolution module to achieve efficient feature extraction and representation. The adaptive graph convolution block integrates the graph convolution network (Grapher) and the feedforward neural network (FFN). The structural diagram is shown in the figure. Figure 3As shown in the figure: The graph convolution network (Grapher) is composed of the first two-dimensional convolution (Conv2D), the first fully connected layer, the second two-dimensional convolution, the second fully connected layer, the parameterized rectified linear unit activation function (PreLu), the maximum correlation graph convolution block (MRGCN), the third two-dimensional convolution (Conv2D), the third fully connected layer and the first path drop block (Droppath). The feedforward neural network part (FFN) is a simple multi-layer perceptron, which is composed of the fourth two-dimensional convolution (Conv2D), the fourth fully connected layer, the parameterized rectified linear unit activation function (PreLu), the fifth two-dimensional convolution (Conv2D), the fifth fully connected layer and the second path drop block (Droppath). Using the feedforward neural network (FFN) for node feature conversion can improve node diversity and alleviate the transition smoothness problem caused by the deepening of the number of layers in the traditional graph neural network. Both the graph convolution network (Grapher) and the feedforward neural network (FFN) use residual connections, which helps to alleviate the gradient vanishing problem in deep networks and improve training efficiency.

[0076] In step S4, the classifier uses the Softmax function to convert the original prediction value (logits) without probability conversion into a probability distribution according to the feature information of the optimal topology output by the topology map construction and update module, and selects the category with the highest probability as the final prediction result. In the present invention, there are two categories of living body and disguised face, and the category with higher probability is determined as the category of the current input image.

[0077] The present invention introduces a meta-learning strategy to improve the adaptability of a robust face liveness detection model, and transforms the traditional training stage of the robust face liveness detection model into two stages: inner loop training and outer loop training. In the inner loop training stage, the meta-learning parameters are updated and optimized to improve the rapid adaptability of the robust face liveness detection model. In the outer loop training stage, the meta-learning parameters adapted by the inner loop training are first loaded, and then the overall parameters of the robust face liveness detection model are updated. Since the classifier is at the tail of the robust face liveness detection model and is most sensitive to the gradient, the present invention selects the classifier parameters as the meta-learning parameters. , and other parameters besides the meta-learning parameters As conventional learning parameters, conventional learning parameters and meta-learning parameters constitute the overall parameters of the robust face liveness detection model. The detailed process is as follows Figure 4 As shown, the support set sample and query set sample , which are respectively applied to the inner loop training and outer loop training stages; the purpose of the inner loop training is to quickly adapt to new tasks, so the objective function of the inner loop training selects the KL divergence loss, and the sensitivity of the KL divergence loss to soft labels helps to quickly adjust the model parameters of the robust face liveness detection model; the purpose of the outer loop training focuses more on refining and optimizing the robust face liveness detection model on a wider range of data, so the objective function of the outer loop training selects the soft label cross entropy loss, which can ensure that the robust face liveness detection model can learn stable feature representations on various data.

[0078] In the inner loop training process, we first select samples from the support set The sampled samples are input into the robust face liveness detection model, and the KL divergence loss of the inner loop training is calculated through forward propagation. The calculation formula is shown in (7). Then the gradient of the KL divergence loss is calculated and back propagation is performed to update the meta-learning parameters ; In the inner loop training, only the meta-learning parameters are updated , which can achieve rapid adaptation and quickly capture the key features of new tasks. The meta-learning parameter update formula in the inner loop training is shown in (8):

[0079] (7);

[0080] (8);

[0081] in, is the KL divergence loss, is the overall parameter, M is the number of categories, y c is the soft label of category c in the target distribution, p c is the probability of class c in the predicted distribution, is the meta-learning parameter after the inner loop training update, is the gradient operator, which represents the meta-learning parameters Seeking guidance, is the learning rate of the inner training loop.

[0082] During the outer loop training process, the goal is to update the overall parameters of the robust face liveness detection model. , optimize the performance of the robust face liveness detection model on the entire dataset so that it can have better generalization ability on new tasks. Before the outer loop parameters are updated, the meta-learning parameters updated by the inner loop training will be loaded first , and then forward propagation is performed on the query set samples to calculate the soft label cross entropy loss , as shown in formula (9), and then with the KL divergence loss of the inner loop training Add them together to get the outer loop loss, and use the outer loop training to update the overall parameters:

[0083] (9);

[0084] (10);

[0085] (11);

[0086] in, It is the loss of the external circulation; is the overall parameter of the tth iteration, is the overall parameter of the t+1th iteration, is the gradient operator, which means the overall parameter Seeking guidance, is the learning rate of the outer loop training, represents parameters other than the meta-learning parameters.

[0087] In summary, in the inner loop training stage, the model's meta-learning parameters can be quickly adjusted through the optimization of the KL divergence loss; and then the soft label cross entropy loss function in the outer loop training stage describes the similarity and uncertainty between categories more delicately, and further accurately updates the overall parameters of the robust face liveness detection model. In addition, this separate learning process of inner loop training and outer loop training can avoid the problem of conflict that may occur when rapid adaptation and fine-tuning are performed simultaneously, and can enable the robust face liveness detection model to quickly adjust and continuously optimize when facing new tasks, thereby improving the adaptability and robustness of the robust face liveness detection model.

[0088] In the testing phase, the test dataset is independent of the training dataset and has been preprocessed as necessary. The best robust face liveness detection model weights saved in the training phase are loaded, and these weights should perform best on the validation set. The prediction results of the robust face liveness detection model are quantitatively analyzed using predetermined evaluation indicators.

[0089] The detection effect of the robust face liveness detection model in the present invention is verified through the following experimental process.

[0090] The hardware environment of the experiment of this invention is NVDIA GeForce RTX 3080 graphics card, the programming language is Python3.8, and the framework uses Pytorch; the face image size input by the robust face liveness detection model is 256 256 3. Before training, you can pre-process the data by flipping, color jittering, random erasing, mixing images, cutting and mixing, etc., and use Mixup or Cutmix methods to enhance the data. The batch size is set to 32, and the default size of the image block for the face image is 4. 4. The face image training samples are divided into two parts: support set samples and query set samples to adapt the robust face liveness detection model. The inner loop training and outer loop training stages after the meta-learning strategy is introduced. The learning rate of the inner loop training stage is set to 0.01, and the learning rate of the outer loop training stage is set to 0.001. The optimizer is Adamw, which is a variant of the Adam optimizer. It combines weight decay (L2 regularization) with the Adam optimizer. The weight decay value is 0.05, the initial learning rate is 2e-3, and it becomes 0.1 times the original after every 30 rounds of training.

[0091] In order to verify the generalization performance of the robust face liveness detection model proposed in this paper, this paper conducts cross-dataset test experiments on four public datasets: OULU-NPU, CASIA-FASD, Replay-Attack, and MSU-MFSD. These four datasets each focus on different attack types, collection environments, and sample complexity, which makes cross-dataset model evaluation challenging. To facilitate recording experimental results, the datasets are abbreviated as O, C, I, and M, and O->I is used to indicate that OULU-NPU is used as the training set and Replay-Attack is used as the test set. In terms of performance evaluation indicators, the half total error rate (HTER) is used as the evaluation indicator, which is the average of the false rejection rate (FRR) and the false acceptance rate (FAR). The results are shown in Table 1. It is compared with classic or latest face liveness detection methods such as face counter-spoofing with learning generalized deception cues (LGSC), face liveness detection with cross-batch storage mechanism metric learning (XBM_ML), application of global semantic enhancement in face camouflage feature extraction (GSE-FAS), cross-domain face representation attack detection based on multi-domain separation representation learning (MDDR), unsupervised adversarial domain adaptation algorithm for cross-domain face representation attack detection (DRUDA), unified unsupervised semi-supervised domain adaptation network (USDAN) for cross-scene face anti-spoofing, generative domain adaptation algorithm (GDA) for face anti-spoofing, and circular disentangled feature transformation (CDFTN) for face anti-spoofing. The experimental results are shown in Table 1. It can be seen from Table 1 that the average HTER of the method proposed in this paper reaches 8.7% in the cross-dataset experiment, which exceeds the classic or current latest liveness detection methods.

[0092] Table 1. Semi-total error rate indicators in cross-dataset comparison experiments

[0093]

[0094] The ablation experiment is used to verify the influence of each component in the robust face liveness detection model on the entire model. The selected components are dynamic K-nearest neighbor dense algorithm, meta-learning strategy, residual connection and feedforward neural network. OULU-NPU is used as the training set and Replay-Attack is used as the test set. The test effect evaluates the performance of the model based on the half total error rate (HTER). The ablation experiment results are shown in Table 2.

[0095] Table 2 The impact of different components on the overall generalization performance of the robust face liveness detection model

[0096]

[0097] As can be seen from Table 2, when any component is removed, the performance of the robust face liveness detection model will decrease, which indicates that all components are conducive to improving the generalization performance of the robust face liveness detection model. Among them, the dynamic K nearest neighbor dense algorithm is responsible for selecting the best K value according to the local density of the data to dynamically construct the topology map. The optimization of the topology map enables the robust face liveness detection model to more accurately capture and express the complex relationship in the data. Therefore, the dynamic K nearest neighbor dense algorithm has the greatest impact on the robust face liveness detection model. The meta-learning strategy mainly plays a role in the training process. By optimizing the initialization of the robust face liveness detection model and quickly adapting to new tasks, the generalization ability and rapid adaptability of the robust face liveness detection model are improved, especially when the number of new samples is limited. The roles of the residual connection and the feedforward neural network are to enable the robust face liveness detection model to effectively transfer gradients in the deep network and enhance the learning ability of the robust face liveness detection model for features. They mainly play an auxiliary role, such as improving the training stability and the quality of feature representation, and have relatively small effects.

[0098] In addition, during the meta-learning optimization process, the robust face liveness detection model uses two soft label loss functions (KL divergence loss function and soft label cross entropy loss) in the inner loop training and outer loop training stages. The combined effects of different soft label loss functions are compared in order to select the best combination. The results are shown in Table 3. This experiment uses OULU-NPU as the training set and Replay-Attack as the test set. The test results of the robust face liveness detection model are evaluated based on the half total error rate (HTER). In Table 3, KLD refers to KL divergence loss and STCE refers to soft label cross entropy loss.

[0099] Table 3 Performance comparison of different combinations of soft label loss functions

[0100]

[0101] As shown in Table 3, (1) when the KLD function is selected in the inner loop training and the STCE function is selected in the outer loop training, the experimental effect is the best. The possible reason is that the sensitivity of KLD to soft labels helps to quickly adjust the parameters of the robust face liveness detection model, while the STCE function has strong fault tolerance and can ensure that the robust face liveness detection model can learn stable feature representations on various data. Therefore, their combination can achieve the best effect. (2) The loss function in the inner loop training stage has the greatest impact on the performance of the robust face liveness detection model, which will directly affect the rapid adaptability of the robust face liveness detection model on the data set. When the KLD function is selected in the inner loop training, the experimental results are better than those when the STCE function is selected.

[0102] In the process of constructing the topological graph, the number of neighbor nodes K is a hyperparameter that controls the aggregation scale. If the value of K is too small, many useful details will be lost; if it is too large, it may cause excessive smoothing of features and a significant increase in time complexity. Therefore, different data sets should choose different numbers of neighbors according to their local densities. This section compares the liveness detection performance of the robust face liveness detection model using the dynamic K nearest neighbor dense algorithm and the traditional K nearest neighbor algorithm through a single data set internal experiment. The results are shown in Table 4. The experiment uses four data sets: CASIA-FASD (C), Replay-Attack (I), OULU (O) ​​and MSU-MFSD (M). The area under the curve (AUC) indicator is used for performance evaluation. The K value set by the traditional K nearest neighbor algorithm refers to the best K value obtained by the visual graph neural network (VIGNN) experiment: 9, while the dynamic K nearest neighbor dense algorithm selects the best K value according to the data point density.

[0103] Table 4 Comparison of detection performance between dynamic K-nearest neighbor dense algorithm and traditional K-nearest neighbor algorithm in internal experiments of data set

[0104]

[0105] As shown in Table 4, the dynamic K nearest neighbor dense algorithm exceeds the detection performance of the traditional K nearest neighbor algorithm on the four data sets, and the average AUC index exceeds 3.1%. It can be seen that the dynamic K nearest neighbor dense algorithm adaptively adjusts the number of neighbor nodes according to the local density changes of the data. Compared with the traditional K nearest neighbor algorithm, the dynamic K nearest neighbor dense algorithm has greater flexibility and can obtain better detection accuracy.

[0106] Another embodiment of the present invention discloses a robust face liveness detection system based on an adaptive graph neural network, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the robust face liveness detection method based on an adaptive graph neural network of the present invention is implemented.

[0107] The above-mentioned robust face liveness detection system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface, whose settings and functions are known in the art and will not be described in detail here.

[0108] Although the present specification has shown and described the embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will think of many changes, modifications and alternatives without departing from the thought and spirit of the present invention. It should be understood that in the process of practicing the present invention, various alternatives to the embodiments of the present invention described herein may be adopted.

[0109] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A robust face liveness detection method based on an adaptive graph neural network, wherein the adopted robust face liveness detection model includes a data processing module, a feature extraction module, a topology map construction and update module and a classifier according to the functional division, and is characterized in that: Here are the steps: S1: The data processing module divides the face image training samples into support set samples and query set samples according to the set ratio, and performs data preprocessing and data enhancement processing on the support set samples and query set samples in batches. Finally, the face image is divided into image blocks of the same size through block processing; S2: vectorize the image blocks through the feature extraction module to extract the facial image features; S3: The topology map construction and update module includes a dynamic K-nearest neighbor dense algorithm and an adaptive graph convolution module. According to the features of the face image, the spatial position information of the image block is introduced, and each divided image block is regarded as a node. The dynamic K-nearest neighbor dense algorithm is used to adaptively construct the topology map. The update of the topology map includes the update of the edge and the update of the node. The node feature is updated by the adaptive graph convolution module. After the node feature is updated, the edge is updated by the dynamic K-nearest neighbor dense algorithm, and the edge update and the node update process are cyclically updated to obtain the optimal topology map. The adaptive graph convolution module is composed of an adaptive graph convolution block and a downsampling layer. A total of twelve adaptive graph convolution blocks are used, which are divided into four parts: 2, 2, 6, and 2. Each part is connected by a downsampling layer, and the feature dimension of each part is continuously increased. The adaptive graph convolution block integrates the graph convolution network and the feedforward neural network. Both the graph convolution network and the feedforward neural network use residual connections. S4: The classifier determines whether it is a living or disguised face based on the feature information of the optimal topological map.

2. The method for robust face liveness detection based on adaptive graph neural network according to claim 1, characterized in that: In step S1, data preprocessing is to randomly select flipping, color jittering, erasing and mixing images to process sample data; Data enhancement processing is to randomly select mixed data enhancement or cut mixed data enhancement method to perform data enhancement.

3. The method for robust face liveness detection based on adaptive graph neural network according to claim 1, characterized in that: The dynamic K nearest neighbor dense algorithm is used to adaptively construct the topology map by converting each image block into a feature vector and using it as a node. Includes node embedding features With position encoding , get an unordered set of nodes ; represents the i-th node, i∈{1,2,…,N}, N is the number of nodes. For each node, the pairwise distance between nodes is calculated by the defined metric function, and then the K nearest neighbor nodes are selected by the dynamic K nearest neighbor dense algorithm. The neighbor node set is expressed as , and for all neighboring nodes , construct connection nodes and nodes Edge , and then get the graph , Represents a set of edges.

4. The method for robust face liveness detection based on adaptive graph neural network according to claim 1, characterized in that: The processing process of the dynamic K nearest neighbor dense algorithm is as follows: First, based on the node characteristics Calculate the pairwise distances between nodes and the average distance from each node to all other nodes , and then calculate the inverse density of each node , and normalize the inverse density; （1）； in, represents the normalized inverse density of each node, is the maximum normalized inverse density, is the minimum normalized inverse density; Take the median to determine the K value of the sample, and first calculate the median benchmark: , and then calculate the K value based on the median basis: （2）； Among them, quantile(•) is a statistical function, Indicates rounding to the nearest integer. is the maximum K value set, is the minimum K value set; Finally, boundary conditions are applied to ensure that the K value is within the set arrive between: （3）。 5. The method for robust face liveness detection based on adaptive graph neural network according to claim 3, characterized in that: In step S3, the update of nodes is to achieve information exchange between nodes by aggregating and updating the embedded features of neighboring nodes: （4）； in, is the updated i-th node, and are the learnable parameters for aggregation and update operations, respectively. and They are aggregation and update functions respectively; the aggregation operation calculates the representation of the node by aggregating the embedded features of neighboring nodes, and the update operation further performs nonlinear transformation on the aggregated embedded features; The adaptive graph convolution module uses maximum correlation graph convolution to capture embedded features. The aggregation and update formulas are as follows: (5) ； (6)； For Node Its neighbor nodes The embedded features obtained by aggregation through maximum correlation graph convolution, is The embedded features obtained after updating; The update process of the entire node is expressed as: , is the updated embedding feature, and AGCN represents the operation of the adaptive graph convolution module.

6. The method for robust face liveness detection based on adaptive graph neural network according to claim 1, characterized in that: The graph convolution network is composed of a first two-dimensional convolution, a first fully connected layer, a second two-dimensional convolution, a second fully connected layer, a parameterized rectified linear unit activation function, a maximum correlation graph convolution block, a third two-dimensional convolution, a third fully connected layer and a first path discard block in sequence.

7. The method for robust face liveness detection based on adaptive graph neural network according to claim 1, characterized in that: In the training phase, the training phase of the robust face liveness detection model is transformed into two phases: inner loop training and outer loop training. The inner loop training phase updates and optimizes the meta-learning parameters. In the outer loop training phase, the meta-learning parameters adapted by the inner loop training are first loaded, and then the overall parameters of the robust face liveness detection model are updated, and the classifier parameters are selected as the meta-learning parameters. .

8. The method for robust face liveness detection based on adaptive graph neural network according to claim 7, characterized in that: The objective function of the inner loop training is KL divergence loss, and the objective function of the outer loop training is soft label cross entropy loss.

9. A robust face liveness detection system based on an adaptive graph neural network, comprising a processor and a memory, wherein the memory stores computer program instructions, characterized in that: When the computer program instructions are executed by the processor, the robust face liveness detection method based on adaptive graph neural network described in any one of claims 1 to 8 is implemented.

Images