A face anti-fraud method and system based on fused differential graph convolution

Abstract

The present invention discloses a face anti-fraud method and system based on fused differential graph convolution, which divides the input image into non-overlapping image blocks, and then regards each image block as a node of a topological map, and finds the nearest neighbor node according to the characteristics of each node to construct a topological map; the obtained topological map is sent to the fused differential graph convolution module to update and aggregate the nodes and edges, and extract fraud features; the fraud features extracted by the fused differential graph convolution module are sent to the pixel-level regression loss function, forcing the fraud features of the living sample to be all 0 images, and no explicit constraints are imposed on the fraud features of the fraud samples, and the regressed fraud features are obtained; finally, the obtained fraud features are superimposed with the features of the image, and then sent to the classifier to calculate the classification probability, and then the category of the image is determined. The present invention can extract subtle, long-distance dependent fraud features; improve the representation ability and credibility of the subtle features of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of face recognition, and in particular relates to a face anti-fraud method and system based on fused differential graph convolution. Background Art

[0002] In the information age, facial information has become everyone's unique security password. Because it contains significant interests, malicious attacks on face recognition systems are endless. For example, attack methods such as photos, videos, 3D masks and AI face-changing have brought major security risks to face information security. Face anti-fraud technology (FAS) is used to deal with fraud in face recognition. It is a self-evolving problem, that is, attack and defense develop iteratively, so FAS always faces new challenges. In particular, with the advancement of science and technology, fraud methods have become more sophisticated and difficult to identify, which increases the complexity of fraud detection; the continuous evolution and diversification of fraud methods also put forward higher requirements on the generalization ability of the model.

[0003] Face anti-fraud detection mainly distinguishes authenticity by analyzing the difference between live faces and fraudulent faces. Early FAS models mostly used distinguishing features such as texture, color, spectrum, skin, and image quality for liveness detection. With the development of deep learning technology, people began to use the flexibility, adaptability, and powerful complex data representation capabilities of deep features, while combining prior knowledge of liveness detection to carry out anti-fraud detection. For example, based on the fact that fraud samples such as photos and video replays do not have real facial depth information, convolutional neural networks are used to extract features from conventional visible light face images and face depth maps recorded by Kinect (a sensor device developed by Microsoft) for classification of live faces and prosthetic faces; in view of the lack of biological activity of fraudulent faces, deep models are used to capture the periodic changes in remote heart rate, and liveness detection is achieved based on the subtle differences in light changes on the skin surface of live and prosthetic faces.

[0004] With the development of FAS technology, the FAS detection results on a single dataset are relatively ideal, but cross-dataset FAS detection still faces great challenges. Therefore, people have introduced technologies such as unilateral domain adversarial generalization and conditional domain adversarial generalization to improve the generalization performance of the model and obtain better detection results. In face images, some entangled features that are weakly correlated or irrelevant to fraud information will seriously hinder the performance of liveness detection. In particular, facial structural features dominate face images and can easily affect the causal relationship of model detection. Therefore, it is necessary to remove irrelevant noise information and separate strongly correlated fraud features from the "entangled body". Therefore, people use disentanglement technology to separate fraud features, introduce metric learning to force compactness within the class and separation between classes, or introduce global semantic enhancement technology to improve the model's ability to extract long-distance dependent fraud features.

[0005] In order to enhance the ability of convolutional neural network (CNN) to extract fine-grained texture information, people introduced center difference convolution (CDCN) on the basis of standard convolution to enhance the representation ability of features. Center difference convolution provides an effective feature representation method for face liveness detection by extracting the intensity and gradient information of the image, enhancing the model's ability to capture fine-grained features and showing good robustness in various environments. However, due to the irregularity, hierarchical structure, and complex relationships of effective semantic information in the image, it is likely to limit the performance of traditional CNN in image processing. Compared with CNN, graph neural network (GNN) usually has a weaker inductive bias and can more flexibly capture irregular and complex object relationships. Therefore, GNN is currently being used more and more widely in image processing. Existing studies have applied multiple graph convolution operations such as maximum correlation graph convolution (MRGCN), GraphSAGE (a graph neural network model for graph node embedding learning) and graph isomorphism convolution (GIN) based on image representation to convert and communicate information between nodes, and have achieved groundbreaking research results in image recognition and object detection tasks. Summary of the invention

[0006] Based on the above prior art, the present invention provides a face anti-fraud method and system based on fused differential graph convolution, which fully utilizes the powerful complex data representation capability and long-distance feature propagation characteristics of GNN to extract subtle, long-distance dependent fraud features; and performs weighted fusion of the center difference graph convolution and the maximum correlation graph convolution to further improve the representation capability of the model's subtle features; and utilizes the topological relationship of GNN to improve the interpretability of the model, so as to improve the credibility of the model.

[0007] The present invention is implemented by the following technical solution: A face anti-fraud method based on fused differential graph convolution, the steps are as follows:

[0008] S1: Divide the input image into non-overlapping image blocks, then regard each image block as a node of the topological map, and find the nearest neighbor node according to the characteristics of each node to construct the topological map;

[0009] S2: Send the obtained topology graph to the fusion differential graph convolution module to update and aggregate the nodes and edges and extract fraud features;

[0010] S3: The fraud features extracted by the fusion differential graph convolution module are fed into the pixel-level regression loss function to force the fraud features of the living samples to be all zero images, while no explicit constraints are imposed on the fraud features of the fraud samples, and the regressed fraud features are obtained;

[0011] S4: Finally, the obtained fraud features are superimposed with the image features and sent to the classifier to calculate the classification probability and then determine the category of the image.

[0012] Further preferably, the fused differential graph convolution module (FDGCN) is composed of several CDGrapher blocks separated by downsampling layers; three downsampling layers are used to connect four groups of CDGrapher blocks of different scales, each CDGrapher block is composed of a graph convolution network part and a feedforward neural network part, the graph convolution network part includes three two-dimensional convolutions (Conv2D), three fully connected layers, fused differential graph convolutions, PReLu activation function and DropPath block, the first two-dimensional convolution (Conv2D) and the second two-dimensional convolution are connected using the first fully connected layer, the second two-dimensional convolution and the fused differential graph convolution are connected using the second fully connected layer and the PReLu activation function is adopted, the fused differential graph convolution is connected to the third two-dimensional convolution, the third two-dimensional convolution is connected to the DropPath block through the third fully connected layer, the output of the DropPath block is the output of the graph convolution network part, and the fused differential graph convolution is composed of the maximum correlation graph convolution (MRGCN) and the center differential graph convolution connected in series.

[0013] Further preferably, the feedforward neural network part includes two two-dimensional convolutions, a fully connected layer is set after each two-dimensional convolution, the first fully connected layer is processed by a PReLu activation function, and then enters the second two-dimensional convolution, and the second fully connected layer is connected to a DropPath block.

[0014] Further preferably, the topology is defined as ,in is the set of nodes in the topology graph. is the set of edges in the topological graph. The image blocks are used as nodes of the topological graph. The face image is divided into N non-overlapping image blocks. Each image block is converted into features and forms a node. Then we get , is the feature of the i-th node, i∈{1,2,…,N}, N is the number of nodes, where , is the characteristic dimension of the node;

[0015] When constructing the topology graph, the node characteristics and the node spatial position information Pos are comprehensively considered and summed up: , get the node features, and then according to the node features The distance between nodes is calculated, and the K nearest neighbor algorithm is applied to obtain the K nearest neighbor nodes of each node to form a topological graph represented by the adjacency matrix. The constructed topological graph is defined as: .

[0016] Further preferably, the spatial position information of the node is defined by relative position coding, and the calculation formula of the relative position coding is as follows:

[0017] ;

[0018] ;

[0019] ;

[0020] In the formula, and Respectively represent the horizontal and vertical coordinates of each pixel in the topological map, is the one-dimensional relative position encoding in the x-axis direction, It is the one-dimensional relative position encoding in the y-axis direction. It is a sequence with uniform values ​​between 0 and 1, and its length is the position feature dimension d.

[0021] Further preferably, the aggregation and update operation of the maximum correlation graph convolution is as shown in the formula:

[0022] ;

[0023] ;

[0024] In the formula, represents the set of nearest neighbor nodes of the i-th node, represents the updated feature of the i-th node, represents the feature of the i-th node after aggregation, is the feature of the jth nearest neighbor node of the i-th node, the i-th node is the central node, The function is to find the maximum difference between the central node and the nearest neighbor node. is the concatenation function of the features; and Represents aggregation function and update function respectively; and are the learnable parameters during aggregation and update, respectively;

[0025] The maximum correlation graph convolution (MRGCN) update phase introduces a multi-head update mechanism to promote feature diversification; first Divided into Head , each head is updated in parallel with the new weight, and finally each head is connected to calculate the final update result , and its calculation formula is as follows:

[0026] ;

[0027] in, They represent the 1st, 2nd, …, hth heads respectively. They represent the learnable parameters when the 1st, 2nd,…,hth heads are updated respectively.

[0028] Further preferably, the central difference graph convolution subtracts the features of each nearest neighbor node from the features of the central node, and updates the features of the central node through attention weighted summation. The features of the i-th node ,in, is the batch size, is the number of channels, is the number of nodes, is the number of nearest neighbors, represents the feature difference between the i-th node and its j-th nearest neighbor node, is the weighted feature difference, It means that pixel multiplication is performed on the channel dimension, and finally the weighted feature differences are summed to obtain the features of the aggregated nodes of the central difference graph convolution. , the specific calculation formula is as follows:

[0029] ;

[0030] in, is the learnable channel attention weight.

[0031] Further preferably, the aggregation function calculation formula of the fused differential graph convolution is as follows:

[0032] ;

[0033] In the formula, It is the weight factor that controls the ratio of maximum correlation graph convolution to center difference graph convolution during the update process.

[0034] Further preferably, the mean absolute value loss (MAE) is used as the pixel-level regression loss function for training the fused differential graph convolution module.

[0035] Further preferably, a soft label cross entropy loss function is used as the classification loss function of the classifier.

[0036] The present invention also provides a system for implementing a face anti-fraud method based on fused differential graph convolution, comprising:

[0037] The data preprocessing module divides the input image into non-overlapping image blocks;

[0038] A graph construction module is used to regard each image block as a node of a topological graph and to find the nearest neighbor node according to the characteristics of each node to construct a topological graph;

[0039] Fusion differential graph convolution module to update and aggregate nodes and edges to extract fraud features;

[0040] The diagnosis module is used to superimpose the obtained fraud features and image features, and then send them into the classifier to calculate the classification probability and then determine the category of the image.

[0041] The beneficial effects of the present invention are as follows: the powerful complex data representation capability and long-distance feature propagation characteristics of GNN are utilized to extract subtle, long-distance dependent fraud features, so as to more accurately find the subtle differences between real faces and fraudulent faces; the center difference graph convolution and the maximum correlation graph convolution are weightedly fused to further improve the representation capability of the model's subtle features. This method can not only improve the representation capability of the model, but also use the center difference to eliminate the interference of irrelevant features, thereby improving the accuracy of model recognition; the topological relationship of GNN is used to improve the interpretability of the model to improve the credibility of the model. As concerns about privacy and transparency increase, improving the interpretability of the model not only helps to enhance users' trust in the system, but also promotes technological improvements by revealing the decision logic of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of the face anti-fraud method based on the fused differential graph convolution module of the present invention.

[0043] Figure 2 This is a structural block diagram of the face anti-fraud system based on the fused differential graph convolution module.

[0044] Figure 3 Schematic diagram of the structure of the fused differential graph convolution module.

[0045] Figure 4 Schematic diagram of central difference graph convolution.

[0046] Figure 5 is the weight factor The impact of the value on the test results.

[0047] Figure 6 It is the sample distribution of the iteration number epoch=1 in the iteration process.

[0048] Figure 7 This is the sample distribution of the iteration number epoch=300 in the iteration process. DETAILED DESCRIPTION

[0049] The present invention will be further described in detail below with reference to the accompanying drawings.

[0050] Reference Figure 1 and Figure 2, a face anti-spoofing method based on fused differential graph convolution, the steps are as follows:

[0051] S1: Divide the input image into non-overlapping image blocks, then regard each image block as a node of the topological map, and find the nearest neighbor node according to the characteristics of each node to construct the topological map;

[0052] S2: Send the obtained topology graph to the fusion differential graph convolution module to update and aggregate the nodes and edges and extract fraud features;

[0053] S3: The fraud features extracted by the fusion differential graph convolution module are fed into the pixel-level regression loss function to force the fraud features of the living samples to be all zero images, while no explicit constraints are imposed on the fraud features of the fraud samples, and the regressed fraud features are obtained;

[0054] S4: Finally, the obtained fraud features are superimposed with the image features and sent to the classifier to calculate the classification probability and then determine the category of the image.

[0055] like Figure 3As shown in the figure, the fusion differential graph convolution network (FDGCN) is mainly composed of several CDGrapher blocks separated by downsampling layers. The fusion differential graph convolution module uses three downsampling layers to connect four groups of CDGrapher blocks of different scales, thus generating a pyramid network structure, that is, as the number of network layers increases, the feature dimension continues to increase: {80, 160, 400, 640}, and the corresponding output feature map sizes are H / 4×W / 4, H / 8×W / 8, H / 16×W / 16 and H / 32×W / 32, where H×W is the size of the original input image, H is the height, and W is the width. This pyramid structure can not only improve the accuracy of the FDGCN model, but also better obtain some significant semantic features of the image, and can effectively obtain different scale features of fraud. Each CDGrapher block consists of a graph convolution network part and a feedforward neural network part. The graph convolution network part includes three two-dimensional convolutions (Conv2D), three fully connected layers, fused differential graph convolutions, PReLu activation functions, and DropPath blocks. The first two-dimensional convolution (Conv2D) and the second two-dimensional convolution are connected using the first fully connected layer, the second two-dimensional convolution and the fused differential graph convolution are connected using the second fully connected layer and the PReLu activation function, the fused differential graph convolution is connected to the third two-dimensional convolution, and the third two-dimensional convolution is connected to the DropPath block through the third fully connected layer. The output of the DropPath block is the output of the graph convolution network part. The fused differential graph convolution is composed of the maximum correlation graph convolution (MRGCN) and the center differential graph convolution in series. The feedforward neural network part includes two two-dimensional convolutions, and a fully connected layer is set after each two-dimensional convolution. After the first fully connected layer is processed, it is processed by the PReLu activation function, and then enters the second two-dimensional convolution. The second fully connected layer is connected to the DropPath block. The PReLu activation function is used because it has a learnable parameter that can fine-tune the activation function according to the learning rate to solve the problem of gradient vanishing. It improves the FDGCN model fitting with almost zero additional computational cost and very small risk of overfitting. DropPath is a regularization method that randomly removes some modules in the training batch to reduce the occurrence of overfitting and achieve domain generalization. Feedforward neural networks have powerful feature conversion capabilities when processing unstructured data. Adding this structure can alleviate the over-smoothing problem caused by graph neural networks.

[0056] In this embodiment, the topology map is defined as ,in is the set of nodes in the topology graph. is the set of edges in the topological graph. The topological graph can be regarded as a collection of nodes and edges. For grid-like image data, pixels, superpixels, image blocks or the entire image can be used as nodes of the topological graph. In order to reduce the amount of calculation and retain more feature information, the present invention uses image blocks as nodes of the topological graph. The face image is divided into N non-overlapping image blocks. Each image block is converted into features and forms a node, and then , is the feature of the i-th node, i∈{1,2,…,N}, N is the number of nodes, where , is the feature dimension of the node.

[0057] When constructing the topological graph, the present invention comprehensively considers the characteristics of the nodes and the spatial position information Pos of the nodes and sums them up: , get the node features, and then according to the node features The distance between nodes is calculated, and the K nearest neighbor algorithm is applied to obtain the K nearest neighbor nodes of each node to form a topological graph represented by the adjacency matrix. The constructed topological graph is defined as: .

[0058] Introducing spatial position information into node features enables the FDGCN model to give priority to spatially adjacent nodes when selecting neighbors, which can fully utilize the spatial information of nodes without being constrained by the spatial position of nodes. The node features are learnable features, and the spatial position information of nodes is defined by relative position encoding. The relative position encoding in the present invention is a fixed encoding based on node coordinates (each node is a pixel in the feature map). The calculation formula of relative position encoding is as follows:

[0059] (1);

[0060] (2);

[0061] (3);

[0062] In the formula, and Respectively represent the horizontal and vertical coordinates of each pixel in the topological map, is the one-dimensional relative position encoding in the x-axis direction, It is a one-dimensional relative position encoding in the y-axis direction, and the spatial position information is their splicing. It is a sequence with uniform values ​​between 0 and 1, and its length is the position feature dimension d. Sine and cosine functions of different powers can produce codes of different frequencies, which can capture spatial position information of different scales.

[0063] The update of the topological graph needs to go through the graph convolution layer to aggregate the features of their nearest neighbor nodes. The general operation of graph convolution can be expressed as:

[0064] (4);

[0065] In the formula, This is the topology diagram before updating. is the updated topology map. and are the learnable parameters during aggregation and update, respectively. and They represent aggregation function and update function respectively. Different graph convolution operations can use different aggregation or update methods.

[0066] Through liveness detection experiments on the public OULU-NPU and Replay-Attack datasets, the present invention compares four graph convolution operations: maximum correlation graph convolution (MRGCN), GraphSAGE, Edge GraphConv and graph isomorphism convolution (GIN). The comparison indicator uses the area under the curve (AUC). The experimental results are shown in Table 1. It can be seen from Table 1 that the operation result of the maximum correlation graph convolution far exceeds the other three graph convolutions. Therefore, the present invention selects the maximum correlation graph convolution as the baseline algorithm and makes improvement research on this basis.

[0067] Table 1 Performance of different graph convolutions

[0068]

[0069] The aggregation and update operations of MRGCN are shown in the formula:

[0070] (5);

[0071] (6);

[0072] In the formula, represents the set of nearest neighbor nodes of the i-th node, represents the updated feature of the i-th node, represents the feature of the i-th node after aggregation, is the feature of the jth nearest neighbor node of the i-th node, the i-th node is the central node, The function is to find the maximum difference between the central node and the nearest neighbor node. It is the concatenation function of the features. Aggregation function It is a maximum correlation function, which first calculates the maximum difference between the features of the nearest neighbor node and the features of the central node, and then splices the maximum difference to the features of the central node to obtain a new feature, which is then linearly transformed and then residually connected to the features of the node. This aggregation method helps to introduce important change information of the local neighborhood when updating node features, and residual skip connections can also alleviate the gradient vanishing problem in graph convolution.

[0073] The maximum correlation graph convolution (MRGCN) update phase introduces a multi-head update mechanism to promote feature diversification. Divided into Head , each head is updated in parallel with the new weight, and finally each head is connected to calculate the final update result , and its calculation formula is as follows:

[0074] (7);

[0075] in, They represent the 1st, 2nd, …, hth heads respectively. They represent the learnable parameters when the 1st, 2nd,…,hth heads are updated respectively.

[0076] In addition, in order to accurately capture the relationship between topological graph nodes, the maximum relevance graph convolution (MRGCN) uses a dynamic update method to respond to the update of topological graph nodes in real time, that is, to update the topological graph in time after the node features are updated.

[0077] The maximum correlation graph convolution selects the maximum difference to update the central node. This method helps to introduce important change information in the local neighborhood, but it may also lead to over-abstraction of information, thereby ignoring some subtle features. To this end, the present invention proposes a central difference graph convolution operation to further improve the ability of the FDGCN model to extract subtle fraud features.

[0078] The calculation process of the central difference graph convolution is as follows Figure 4 As shown in the figure. The central difference graph convolution is to subtract the features of each nearest neighbor node from the features of the central node, and update the features of the central node through the weighted summation of attention. Figure 4 In the example, the feature of the i-th node is ,in, is the batch size, is the number of channels, is the number of nodes, is the number of nearest neighbors, represents the feature difference between the i-th node and its j-th nearest neighbor node, is a learnable channel attention weight, whose dimension is , is the weighted feature difference, It means that pixel multiplication is performed on the channel dimension, and finally the weighted feature differences are summed to obtain the features of the aggregated nodes of the central difference graph convolution. , the specific calculation formula is as follows:

[0079] (8);

[0080] The central difference graph convolution can improve the robustness of the FDGCN model to noise and outliers by introducing relative features between nodes, and can better aggregate the features of the central node with the features of the surrounding nearest neighbor nodes.

[0081] Figure 4 middle, These are the features of the 1st, 2nd, 3rd, 4th, 5th, 6th, 7th, 8th, and 9th nodes respectively. The nearest neighbor nodes of the 3rd node are the 2nd, 4th, 5th, and 6th nodes. The feature differences between the 3rd node and the 2nd node are calculated respectively. , the feature difference between the third node and the fourth node , the feature difference between the 3rd node and the 5th node , the characteristic difference between the 3rd node and the 6th node , after weighting, the feature difference between the weighted third node and the second node is obtained , the weighted feature difference between the third and fourth nodes , the weighted feature difference between the third and fifth nodes , the weighted feature difference between the third and sixth nodes , aggregate to get the features of the third node .

[0082] In comparison, the maximum correlation graph convolution can introduce important change information of the local neighborhood, but it is easy to ignore the detail information; while the central difference graph convolution is relatively more focused on the extraction of detail information. To this end, the present invention weightedly fuses the maximum correlation graph convolution with the central difference graph convolution in order to obtain a more comprehensive perspective and improve the robustness, flexibility and adaptability of the FDGCN model. The aggregation function calculation formula of the fused difference graph convolution is as follows:

[0083] (9);

[0084] In the formula, It is the weight factor that controls the ratio of maximum correlation graph convolution to central difference graph convolution during the update process. The larger its value, the greater the influence of central difference graph convolution.

[0085] Combining the above fused differential aggregation operation with the multi-head update operation, the resulting fused differential graph convolution operation can be abbreviated as:

[0086] (10);

[0087] Among them, FDGCN represents the fused differential graph convolution operation.

[0088] Based on the prior knowledge of liveness detection, the present invention assumes that live samples have no fraudulent features. Ideally, the fraudulent features generated by the fused differential graph convolution module should be all zeros. To this end, a pixel-level regression loss is added after the output of the fused differential graph convolution module, and the live samples are forced to tend to the ideal situation through a supervised method. The present invention uses the mean absolute value loss (MAE), that is, the L1 loss, as the pixel-level regression loss function. , which is expressed as follows:

[0089] (11);

[0090] In the formula, Indicates Fraud features learned from live samples, is the number of live samples in the batch. The mean absolute error can avoid the problem of errors offsetting each other, and thus can accurately reflect the size of the actual prediction error.

[0091] In order to further improve the generalization ability of the FDGCN model, the present invention uses the soft label cross entropy loss function as the classification loss function. The traditional cross entropy loss function assumes that the label is a one-hot hard-coded value, and its value is a binary 0 or 1; while the label of the soft label cross entropy loss function is the probability distribution of the sample, and its value is a floating point number between [0,1]. This allows a certain degree of uncertainty in the label and can reduce the overfitting of the FDGCN model to the training data. In order to obtain soft labels, the Mixup and CutMix methods are randomly used for data enhancement. Mixup is to mix the i-th image and the j-th image in a batch according to formula (12), and then generate the corresponding soft label according to formula (13):

[0092] (12);

[0093] (13);

[0094] In the formula, is a randomly generated mixing factor, are two different images in a batch and images The mixed image, and is a unique hot label, is the generated soft label. The CutMix method is similar to the Mixup method. The difference is that CutMix converts the first images The image blocks of the part are stitched into the On the image, the soft label is generated according to formula (13).

[0095] Calculate cross entropy loss based on soft labels The mathematical representation of is as follows:

[0096] (14);

[0097] In the formula, is the number of categories, is the probability of the c-th soft label, is the predicted probability of the cth class. In liveness detection, Usually equal to 2. Soft label cross entropy provides a more delicate way to express the similarity and uncertainty between categories, thereby reducing the impact of incorrect labels to a certain extent and effectively improving the robustness and adaptability of the FDGCN model.

[0098] The above two loss functions are weighted and fused, and the weighting factor is used to balance the proportion of the loss function to maximize the performance. The resulting comprehensive loss function is The calculation formula is as follows:

[0099] (15);

[0100] In the formula, and are two custom weighting factors, which are set to 0.4 and 0.6 respectively in the experiment of the present invention.

[0101] In order to extract discriminative fraud features from complex data, the FDGCN model first needs to build a topological map from the image, and then apply fused differential graph convolution to transform and communicate information between nodes to extract subtle, global fraud features. Finally, the FDGCN model is optimized through pixel-level regression loss and soft label cross entropy loss to improve the detection accuracy and robustness of the FDGCN model.

[0102] The hardware environment of the experiment is NVDIA GeForce RTX 2080Ti graphics card, the programming language is Python3.8, and the framework is Pytorch. Data preprocessing uses data enhancement methods such as random cropping, horizontal flipping, and color jitter to ensure that the positive and negative samples maintain a 1:1 ratio, reduce data imbalance and overfitting, and use Mixup or CutMix methods to generate soft labels for samples. In the training phase, pytorch's built-in mixed precision is used for training, the batch size is set to 32, the weight decay is 0.5, the training batch is 300, and the AdamW optimizer is used to optimize the network, the learning rate is set to 2e-3, the node dropout rate is 0.2, and the dropout-path rate is set to 0.3.

[0103] Four datasets, MSU-MFSD, CASIA-MFSD, Replay-Attack and OULU-NPU, were used in the experiment. MSU-MFSD and CASIA-MFSD mainly include two common attack types: printing attack and replay attack; CASIA-MFSD also has some mask attacks, which is to make the tester wear a simulation mask to disguise the face; Replay-Attack focuses on replay attacks in facial recognition systems, and there is also a cut photo attack, which is to cut the printed photo and place it on the face to disguise the face; OULU-NPU also includes printing attacks and video replay attacks, but because the dataset is recorded with a mobile phone, it derives four protocols for different scenarios and mobile phone types, namely Protocol 1: Different scenarios; Protocol 2: Different attack methods; Protocol 3: Different mobile phones; Protocol 4: Different scenarios + attack methods + mobile phones.

[0104] This experiment uses four protocols of the OULU-NPU dataset to conduct tests and compares them with well-known models in the field and the latest models. The models involved in the test are the Learning Common Fraud Clues Model (LGSC), Cross-Batch Storage Metric Learning Model (XBM_ML), Global Semantic Enhancement Model (GSE-FAS), Center Difference Convolution Model (CDCN), CDCN++, Image Patch Model PatchNet, Spatiotemporal Anti-Fraud Model (STASN) and Auxiliary, as well as the latest Embedding Level and Prediction Level Consistency Regularization Model (EPCR) and Statistical Adapter-Tag Style Regularization (S-Adapter-TSR). The experimental evaluation indicators are: True Face Classification Error Rate (BPCER), Fraud Face Classification Error Rate (APCER) and Average Classification Error Rate (ACER). The experimental results are shown in Table 2.

[0105] Table 2 Comparative experiments on the OULU dataset

[0106]

[0107] As can be seen from Table 2, the comprehensive performance of the FDGCN model proposed in the present invention has obvious advantages over other algorithms. Specifically, the FDGCN model has an APCER of 0 in protocol 1, and the best results of BPCER and ACER in protocol 2 are 0.2 and 0.5 respectively. The performance in protocol 3 is not optimal, but the performance in protocol 4 is good, and both BPCER and ACER have achieved the best results. The experimental results show that the present invention is not inferior to well-known models and the latest models in detection accuracy.

[0108] (3) Cross-dataset comparative experiments

[0109] Table 3 HTER indicators tested across datasets

[0110]

[0111] In order to further verify the advantages of the FDGCN model in generalization ability, the present invention conducted cross-dataset experiments on CASIA-MFSD, Replay-Attack and Replay-Attack. The generalization ability of the model was verified by comparing STASN, Auxiliary, LGSC, CDCN, CDCN++ and PatchNet. The half total error rate (HTER) and area under the curve (AUC) were used for evaluation in the experiment, and the experimental results are shown in Table 3 above.

[0112] It can be seen that when CASIA-MFSD is used as the training set and Replay-Attack is used as the test set, the HTER index of the FDGCN model proposed in the present invention is 1.4 worse than that of CDCN++, while when Replay-Attack is used as the training set and CASIA-MFSD is used as the test set, the FDGCN model performs best, with an HTER index of only 15.2. When the high-resolution dataset (CASIA-MFSD) is used for model training and the low-resolution dataset (Replay-Attack) is used for testing, the performance of the FDGCN model does not reach the best state. This is because high-resolution images can provide more detailed texture information for the model, and this information is often incomplete or missing in low-resolution images, so the feature information obtained during training cannot be fully utilized during testing. On the contrary, the fraud behavior features learned from low-resolution images can be better transferred to the recognition of high-resolution images by the FDGCN model. Although low-resolution images may not be clear enough in some details, the fraud features they contain still have certain applicability and recognition value in images with higher resolution.

[0113] In order to further verify the generalization performance of the FDGCN model, the present invention also conducted cross-domain experiments on multiple datasets. Specifically, three of the four datasets were selected as training domains, and the remaining datasets were used as test domains to compare the effects of domain generalization. In this experiment, the four datasets MSU-MFSD, CASIA-MFSD, Replay-Attack, and OULU-NPU were replaced by the letters M, C, I, and O, respectively, to derive four experiments, namely O&C&I→M, O&C&M→I, O&M&I→C, and M&C&I→O. The experimental indicators used HTER and AUC. In this experiment, the present invention will be compared with the single-side domain generalization model (SSDG), PatchNet, regularized fine-grained original learning model (RFM), and domain dynamic adjustment meta-learning model (D 2 AM), Neural Architecture Search Model (NAS-FAS), Generative Domain Adaptation Model (GDA) and Energy-based Domain Generalization (EBDG) are compared with the latest research such as Chaotic Assembly Network (SSAN), Instance-aware Domain Generalization Model (IADG) and Language Image Pre-training Multimodal Contrastive Learning (FLIP-MCL). The experimental results are shown in Table 4:

[0114] Table 4 Comparative experimental results of multiple datasets

[0115]

[0116] It can be seen that the performance of the FDGCN model proposed in this invention is comparable to that of the FLIP-MCL model, and it has a clear leading advantage over other models. In the three experiments of O&C&M→I, O&C&I→M, and O&C&I→M, the performance index of the FDGCN model exceeds that of the FLIP-MCL model. In the O&M&I→C experiment, the performance index of the FDGCN model is not as good as that of the FLIP-MCL model.

[0117] After experimental comparisons of single episodes and cross-datasets, the FDGCN model proposed in the present invention has a more obvious advantage in detection accuracy compared with other models. The following experiments further compare the computational complexity of the models. The experimental indicators used are floating-point operations (FLOPs). The results are shown in Table 5. It can be seen from the table that the number of floating-point operations generated by the forward propagation of the FDGCN model is 6.84G, second only to the PatchNet model, and far superior to the CDCN++ (Center Differential Convolution++) model. It can be seen that the computational complexity of the FDGCN model is relatively low and has certain competitiveness. In the single data experiment (see Table 2), the detection accuracy of the FDGCN model proposed in the present invention is better than the comparison model, that is, the FDGCN model can obtain higher detection performance with lower computational complexity.

[0118] Table 5 Comparison of model calculation amount

[0119]

[0120] The present invention combines the maximum correlation graph convolution with the center difference graph convolution to improve the graph convolution operation. The value is used to adjust the impact of the central difference graph convolution on the graph convolution operation. For details, see formula (9). The larger the value, the greater the weight of the central difference in the graph convolution, and vice versa. In order to choose the best The experiment on the OULU-NPU dataset verifies the different The value has an impact on the detection performance ACER. The results are as follows Figure 5 As shown. It can be seen that with As the value increases, the value of ACER decreases, and the detection effect of the model becomes better. When , the model effect reaches the best, however, if we increase value, the ACER value begins to increase, and the model performance decreases. Therefore, each experiment of the present invention is set .

[0121] Table 5 Ablation experiment test under OULU-NPU protocol 1

[0122]

[0123] In order to verify the effectiveness and necessity of the FDGCN model proposed in this paper, an ablation experiment was carried out on the protocol 1 of OULU-NPU. In each experiment, some modules or methods were removed and compared with the complete model. Specifically, the original graph convolution was used first, and then MRGCN, FDGCN, residual structure and random path inactivation were added in turn, and finally compared with the FDGCN model. The experimental results are shown in Table 5 above.

[0124] It can be seen from the table that the effect of the original graph convolution is not ideal, and its performance index is not as good as the index value of adding any one of the maximum correlation graph convolution, center difference graph convolution, and residual structure. In the modules added separately, the effect of center difference graph convolution is better than other methods. When more than two modules are added (excluding random path deactivation), the effect is slightly improved compared to when there is only one module. When three modules are added but no random path deactivation is performed, the effect is already ideal, but the BPCER is still high. In the final stage, random path deactivation is added, which is the final model of the present invention. It can be seen that the effect is the best, with APCER of 0, BPCER of 0.3, and ACER of 0.1. In summary, the improved methods proposed in the present invention have contributed to the improvement of model performance.

[0125] The T-SNE graph is used to show the distribution of live samples and prosthetic samples during the iteration process. The experiment uses the OULU-NPU dataset. The dots represent live samples, and the triangles represent prosthetic samples. The results are shown in Figure 2. Figure 6 and Figure 7 As shown in the figure, at the beginning of training, when the iteration number epoch=1, the live samples and the prosthetic samples are randomly distributed in the feature space; but as the training progresses, when the iteration number epoch=300, the live samples and the prosthetic samples have been separated, the live samples are clustered together, and the prosthetic samples are far away from the prosthetic samples, which proves that this model can promote the compactness between live samples and the separation between prosthetic samples and live samples.

[0126] Another embodiment of the present invention provides a system for implementing a face anti-spoofing method based on fused differential graph convolution, comprising:

[0127] The data preprocessing module divides the input image into non-overlapping image blocks;

[0128] A graph construction module is used to regard each image block as a node of a topological graph and to find the nearest neighbor node according to the characteristics of each node to construct a topological graph;

[0129] Fusion differential graph convolution module to update and aggregate nodes and edges to extract fraud features;

[0130] The diagnosis module is used to superimpose the obtained fraud features and image features, and then send them into the classifier to calculate the classification probability and then determine the category of the image.

[0131] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0132] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A face anti-fraud method based on fused differential graph convolution, characterized in that: Here are the steps: S1: Divide the input image into non-overlapping image blocks, then regard each image block as a node of the topological map, and find the nearest neighbor node according to the characteristics of each node to construct the topological map; S2: Send the obtained topology graph to the fusion differential graph convolution module to update and aggregate the nodes and edges and extract fraud features; S3: The fraud features extracted by the fusion differential graph convolution module are fed into the pixel-level regression loss function to force the fraud features of the living samples to be all zero images, while no explicit constraints are imposed on the fraud features of the fraud samples, and the regressed fraud features are obtained; S4: Finally, the obtained fraud features are superimposed with the image features and then sent to the classifier to calculate the classification probability and then determine the category of the image; The fused differential graph convolution module is composed of several CDGrapher blocks separated by downsampling layers; three downsampling layers are used to connect four groups of CDGrapher blocks of different scales, each CDGrapher block is composed of a graph convolution network part and a feedforward neural network part, the graph convolution network part includes three two-dimensional convolutions, three fully connected layers, fused differential graph convolutions, PReLu activation function and DropPath block, the first two-dimensional convolution and the second two-dimensional convolution are connected using the first fully connected layer, the second two-dimensional convolution and the fused differential graph convolution are connected using the second fully connected layer and the PReLu activation function is adopted, the fused differential graph convolution is connected to the third two-dimensional convolution, the third two-dimensional convolution is connected to the DropPath block through the third fully connected layer, the output of the DropPath block is the output of the graph convolution network part, and the fused differential graph convolution is composed of the maximum correlation graph convolution and the center differential graph convolution connected in series.

2. The face anti-fraud method based on fused differential graph convolution according to claim 1 is characterized in that: The feedforward neural network part includes two two-dimensional convolutions. A fully connected layer is set after each two-dimensional convolution. The first fully connected layer is processed by the PReLu activation function and then enters the second two-dimensional convolution. The second fully connected layer is connected to the DropPath block.

3. The face anti-fraud method based on fused differential graph convolution according to claim 1 is characterized in that: The topology is defined as ,in is the set of nodes in the topology graph. is the set of edges in the topological graph. The image blocks are used as nodes of the topological graph. The face image is divided into N non-overlapping image blocks. Each image block is converted into features and forms a node. Then we get , is the feature of the i-th node, i∈{1,2,…,N}, N is the number of nodes, where , is the characteristic dimension of the node; When constructing the topology graph, the node characteristics and the node spatial position information Pos are comprehensively considered and summed up: , get the node features, and then according to the node features The distance between nodes is calculated, and the K nearest neighbor algorithm is applied to obtain the K nearest neighbor nodes of each node to form a topological graph represented by the adjacency matrix. The constructed topological graph is defined as: .

4. The face anti-fraud method based on fused differential graph convolution according to claim 3 is characterized in that: The spatial position information of the node is defined by relative position coding. The calculation formula of relative position coding is as follows: ； ； ； In the formula, and Respectively represent the horizontal and vertical coordinates of each pixel in the topological map, is the one-dimensional relative position encoding in the x-axis direction, It is the one-dimensional relative position encoding in the y-axis direction. It is a sequence with uniform values ​​between 0 and 1, and its length is the position feature dimension d.

5. The face anti-fraud method based on fused differential graph convolution according to claim 3 is characterized in that: The aggregation and update operation of the maximum correlation graph convolution is shown in the formula: ； ； In the formula, represents the set of nearest neighbor nodes of the i-th node, represents the updated feature of the i-th node, represents the feature of the i-th node after aggregation, is the feature of the jth nearest neighbor node of the i-th node, the i-th node is the central node, The function is to find the maximum difference between the central node and the nearest neighbor node. is the concatenation function of the features; and Represents aggregation function and update function respectively; and are the learnable parameters during aggregation and update, respectively; The maximum correlation graph convolution update phase introduces a multi-head update mechanism to promote feature diversification; first Divided into Head , each head is updated in parallel with the new weight, and finally each head is connected to calculate the final update result , and its calculation formula is as follows: ； in, They represent the 1st, 2nd, …, hth heads respectively. They represent the learnable parameters when the 1st, 2nd,…,hth heads are updated respectively.

6. The face anti-fraud method based on fused differential graph convolution according to claim 3 is characterized in that: The central difference graph convolution subtracts the features of each nearest neighbor node from the features of the central node, and updates the features of the central node through attention weighted summation. The features of the i-th node ,in, is the batch size, is the number of channels, is the number of nodes, is the number of nearest neighbors, represents the feature difference between the i-th node and its j-th nearest neighbor node, is the weighted feature difference, It means that pixel multiplication is performed on the channel dimension, and finally the weighted feature differences are summed to obtain the features of the aggregated nodes of the central difference graph convolution. , the specific calculation formula is as follows: ； in, is the learnable channel attention weight.

7. The face anti-fraud method based on fused differential graph convolution according to claim 6 is characterized in that: The calculation formula of the aggregation function of fused differential graph convolution is as follows: ； In the formula, It is the weight factor that controls the ratio of maximum correlation graph convolution to center difference graph convolution during the update process.

8. The face anti-fraud method based on fused differential graph convolution according to claim 1, characterized in that: The mean absolute value loss is used as the pixel-level regression loss function for training the fused differential graph convolution module.

9. The face anti-fraud method based on fused differential graph convolution according to claim 1, characterized in that: The soft label cross entropy loss function is used as the classification loss function of the classifier.

10. A system for implementing the face anti-fraud method based on fused differential graph convolution as claimed in claim 1, characterized in that: include: The data preprocessing module divides the input image into non-overlapping image blocks; A graph construction module is used to regard each image block as a node of a topological graph and to find the nearest neighbor node according to the characteristics of each node to construct a topological graph; Fusion differential graph convolution module to update and aggregate nodes and edges to extract fraud features; The diagnosis module is used to superimpose the obtained fraud features and image features, and then send them into the classifier to calculate the classification probability and then determine the category of the image.

Images