Face Liveness Detection Method and System Based on Edge Point Difference Hypergraph Convolution

Abstract

The present invention belongs to the technical field of face recognition, and relates to a face liveness detection method and system based on edge-point differential hypergraph convolution. The input face image is divided into non-overlapping image patches, and the image patches are used to extract features through a convolutional layer; then, the K-nearest neighbor and fuzzy C-means clustering algorithms are used to construct a hypergraph structure according to the distances between the features; the obtained hypergraph structure is fed into the edge-point differential hypergraph convolution to aggregate and update the hyperedge and vertex features, obtaining new vertex features; then, optimization is carried out through an asymmetric triplet loss function, and the optimized vertex features are fed into a classifier to obtain class probabilities and determine the class to which the face image belongs. The present invention uses edge-point differential hypergraph convolution to capture the intricate correlations existing in the image, and combines fuzzy C-means clustering to reduce the computational complexity of the hypergraph, realizing face liveness detection with high detection rate and low error rate.

Description

Technical Field

[0001] The present invention belongs to the technical field of face recognition, and particularly relates to a face liveness detection method and system based on edge-point differential hypergraph convolution. Background Art

[0002] In order to detect and defend against attacks, existing face liveness detection (FAS) solutions include domain adversarial generalization techniques, disentanglement techniques, GNN models, and so on. The emergence of domain adversarial generalization techniques is to alleviate the domain shift phenomenon caused by the different data distributions between the training domain and the test domain, and can improve the generalization of the model. The role of disentanglement techniques is to separate fraud clues from factors such as the environment and facial features of the face to improve the robustness of the model. The GNN model is deployed based on pairwise simple graphs. Although the construction method of this kind of graph can provide additional discriminant information between vertices and improve the robustness of the model, it is limited to simple pairwise relationships, resulting in redundant edges and unnecessary memory and computational overhead. Secondly, a basic limitation of simple graphs is that they uniquely connect two vertices, thus uniquely accommodating pairwise relationships. When it comes to modeling the high-order relationships inherent in images, this shortcoming is obvious. Summary of the Invention

[0003] The purpose of the present invention is to provide a face liveness detection method and system based on edge-point differential hypergraph convolution, which uses hypergraph technology based on edge-point differential hypergraph convolution to capture the intricate correlations existing in images, and at the same time combines improved fuzzy C clustering to reduce the computational complexity of the hypergraph. The hypergraph transcends the limitations of pairwise relationships and effectively integrates complex high-order relationships in a unified hypergraph to achieve face liveness detection with high detection rate and low error rate.

[0004] The present invention is realized through the following technical solutions: A face liveness detection method based on edge-point differential hypergraph convolution, the steps are as follows:

[0005] S1: Divide the input face image into non-overlapping image patches, extract features from the image patches through a convolutional layer; then use the K-nearest neighbor and fuzzy C clustering algorithms to construct a hypergraph structure according to the distance between the features;

[0006] S2: Feed the obtained hypergraph structure into edge-point differential hypergraph convolution to flow information in the vertex-hyperedge-vertex manner, aggregate and update hyperedge and vertex features, and obtain new vertex features. The edge-point differential hypergraph convolution includes edge-point differential vertex convolution and self-attention-based hyperedge convolution. Vertex features are aggregated to hyperedges through edge-point differential vertex convolution, and then adjacent hyperedge features are aggregated to vertices through self-attention-based hyperedge convolution to obtain new vertex features. The edge-point differential vertex convolution consists of edge-point maximum correlation graph convolution and edge-point differential graph convolution. The edge-point maximum correlation graph convolution selects the maximum difference between a hyperedge and its connected vertices to update the hyperedge. The edge-point differential graph convolution subtracts the vertex features related to each hyperedge from the hyperedge features to obtain a difference, multiplies it by the attention weight, and then concatenates it with the hyperedge features. Finally, a skip connection is used to add it to the hyperedge features.

[0007] S3: Feed the obtained new vertex features into an asymmetric triplet loss function for optimization to further promote intra-class compactness and inter-class separation in the feature space.

[0008] S4: Feed the optimized vertex features into a classifier to obtain class probabilities and determine the class to which the face image belongs.

[0009] Specifically, in step S1, first, a face image of size H × W is evenly divided into m image patches. After converting each image patch into a feature vector, a data matrix X = [x1, x2, …, x i , …, x m is obtained, where x i is the feature vector of the i-th image patch, and i = 1, 2, …, m. Then, a learnable position encoding is initialized, and the dimension of the position encoding is the same as that of the feature vector of the image patch. The position encoding is added to the feature vector to introduce the spatial position information of the feature. Then, a hypergraph is constructed using the K-nearest neighbor and fuzzy C-means clustering algorithms, and the nearest neighbors are found according to the Euclidean distance between the feature vectors.

[0010] Specifically, in step S1, using the K-nearest neighbor algorithm, with each vertex as the center, 1 hyperedge is generated for each vertex v using the Euclidean distance of the vertex features. Using the fuzzy C-means clustering algorithm for vertex clustering globally, for each vertex v, S - 1 clusters with the closest cluster center to vertex v are found, so as to construct S - 1 hyperedges for each vertex. Together with the 1 hyperedge formed by the K-nearest neighbor algorithm, S hyperedge sets are formed for each vertex. At the same time, a relevant vertex set is constructed for each hyperedge, thus constructing the hyperedge set of vertices and the vertex set of hyperedges.

[0011] Use the symbol Con(e) to represent the vertex set contained in hyperedge e, and use the symbol Adj(v) to represent the hyperedge set composed of all hyperedges containing vertex v. The formula is as follows:

[0012] ;

[0013] ;

[0014] Where \(n\) and \(s\) are the number of vertices in the hyperedge \(e\) and the number of hyperedges containing the vertex \(v\) respectively, represents the \(n\)th vertex, represents the \(s\)th hyperedge.

[0015] Specifically, the calculation formula of edge-point difference graph convolution is as follows:

[0016] ;

[0017] In the formula, represents the \(i\)th hyperedge, represents the hyperedge \(e\) i contains the vertex set; represents the hyperedge and the difference between the connected vertices ; is the learnable channel attention weight, is the learnable parameter during aggregation, is the weighted edge-point difference, represents pixel multiplication in the channel dimension, and finally the weighted edge-point differences are summed to obtain the aggregated value of the edge-point difference graph convolution, represents the edge-point difference graph convolution operation, is the concatenation function of the feature vectors.

[0018] Specifically, the calculation formula of the edge-point difference vertex convolution operation is as follows:

[0019] ;

[0020] In the formula, is the proportionality factor that controls the influence weights of the edge-point maximum correlation graph convolution and the edge-point difference graph convolution during the update process. represents the edge-point maximum correlation graph convolution calculating the maximum difference between each vertex related to the hyperedge and the hyperedge is the concatenation function of the feature vectors; is the updated hyperedge feature.

[0021] Furthermore, preferably, the self-attention-based hyperedge convolution uses the self-attention mechanism in the hyperedge convolution, generates the weight scores of each hyperedge using a multi-layer perceptron, and the output new vertex feature is the weighted sum of the input hyperedge features.

[0022] Further preferably, the features obtained through edge-point differential vertex convolution and hyperedge convolution are used as new vertex features and input into the K-nearest neighbor and fuzzy C-means clustering algorithms to continuously update the hypergraph structure; finally, by combining the asymmetric triplet loss and the soft-label cross-entropy loss, continuous backpropagation is performed to adjust the parameters to minimize the loss.

[0023] The present invention also provides a system for implementing a face liveness detection method based on edge-point differential hypergraph convolution, including:

[0024] An image segmentation module for dividing the input face image into non-overlapping image patches;

[0025] A feature extraction module for extracting features from the image patches;

[0026] A hypergraph construction module that uses the K-nearest neighbor and fuzzy C-means clustering algorithms to construct a hypergraph structure based on the distances between features;

[0027] A hypergraph update module with built-in edge-point differential hypergraph convolution for aggregating and updating hyperedge and vertex features to obtain new vertex features;

[0028] A classifier for determining the category to which the face image belongs.

[0029] The present invention has the following advantages:

[0030] (1) In face liveness detection, a hypergraph can simultaneously capture the complex interactions between multiple features, such as information on lighting changes, facial texture details, reflections, and shadows. This many-to-many relationship better depicts the subtle differences between real faces and fake faces. The hypergraph structure connects multiple vertices through hyperedges, enabling large-scale information propagation and aggregation in a small number of steps. For face liveness detection, this means that the model can more quickly capture the interrelated information of different parts, improving the detection accuracy and efficiency. By leveraging the powerful complex data representation ability of the hypergraph, the ability to extract subtle features of the liveness model is enhanced, thereby improving the detection performance of the model.

[0031] (2) By using a dynamic hypergraph model to learn and generate a hypergraph, the extracted features can retain the original high-dimensional features of the face as much as possible. And through edge-point differential vertex convolution, the problem of over-abstraction of information is alleviated, greatly enhancing the accuracy of the face liveness detection model.

[0032] (3) When the face test data set is input into the hypergraph face liveness detection model, the asymmetric triplet loss method makes the face features generate obvious class boundaries in the feature space.

[0033] (4) The improved fuzzy C-means clustering algorithm can reduce the computational overhead caused by partial repeated calculations. Description of the Drawings

[0034] Figure 1 Schematic diagram of the method of the present invention.

[0035] Figure 2 Schematic diagram of hyperedge convolution based on self-attention. Specific implementation manners

[0036] The present invention will be further elaborated in detail below with reference to the accompanying drawings.

[0037] Referring to Figure 1 , a face liveness detection method based on edge-point differential hypergraph convolution, the steps are as follows:

[0038] S1: Divide the input face image into non-overlapping image patches, extract features from the image patches through a convolutional layer; then use the K-nearest neighbor and fuzzy C-means clustering algorithms to construct a hypergraph structure according to the distances between the features;

[0039] S2: Feed the obtained hypergraph structure into the edge-point differential hypergraph convolution to flow information in the vertex-hyperedge-vertex manner, aggregate and update the hyperedge and vertex features to obtain new vertex features;

[0040] S3: Feed the obtained new vertex features into an asymmetric triplet loss function for optimization to further promote the compactness within classes and the separation between classes in the feature space;

[0041] S4: Feed the optimized vertex features into a classifier to obtain class probabilities and determine the class to which the face image belongs.

[0042] In step S1, first, a face image with a size of H × W is evenly divided into m image patches. After converting each image patch into a feature vector, a data matrix X = [x1, x2,..., x i ,..., x m is obtained, where x i is the feature vector of the i-th image patch, and i = 1, 2,..., m. Then, a learnable position encoding is initialized, and the dimension of the position encoding is the same as that of the feature vector of the image patch. Add the position encoding to the feature vector to introduce the spatial position information of the feature. Then use the K-nearest neighbor and fuzzy C-means clustering algorithms to construct a hypergraph and find the nearest neighbors according to the Euclidean distances between the feature vectors.

[0043] The present invention uses the K-nearest neighbor algorithm to generate one hyperedge for each vertex v with the Euclidean distance of vertex features centered on each vertex, and the generated hyperedges have obvious locality. For the improvement of the fuzzy C-clustering algorithm: starting from the second layer, the scheme of initializing the clustering center at the beginning of each layer is abandoned, and instead, the method of using the clustering center of the previous layer in each layer is adopted. In this way, during the process of gradual convergence of clustering, some repeated calculations can be reduced. The improved fuzzy C-clustering algorithm can perform vertex clustering globally. For each vertex v, S-1 clusters with the nearest clustering center to vertex v can be found, so as to construct S-1 hyperedges for each vertex. Together with the one hyperedge formed by the K-nearest neighbor algorithm, S hyperedge sets can be formed for each vertex. At the same time, a relevant vertex set is constructed for each hyperedge, so as to construct the hyperedge set of vertices and the vertex set of hyperedges.

[0044] Use the symbol Con(e) to represent the vertex set contained in the hyperedge e, and use the symbol Adj(v) to represent the hyperedge set composed of all hyperedges containing the vertex v. The formula is as follows:

[0045] ;

[0046] ;

[0047] where n and s are the number of vertices in the hyperedge e and the number of hyperedges containing the vertex v respectively, represents the nth vertex, represents the sth hyperedge.

[0048] The edge-point differential hypergraph convolution of the present invention includes edge-point differential vertex convolution and self-attention-based hyperedge convolution. Through edge-point differential vertex convolution, vertex features are aggregated into hyperedges, and then through self-attention-based hyperedge convolution, adjacent hyperedge features are aggregated into vertices to obtain new vertex feature embeddings.

[0049] Among them, the edge-point differential vertex convolution is composed of edge-point maximum correlation graph convolution and edge-point differential graph convolution. The edge-point maximum correlation graph convolution selects the maximum difference between the hyperedge and the connected vertices to update the hyperedge. This method helps to introduce important change information in the local neighborhood, but it will ignore some subtle features. The edge-point differential graph convolution focuses on extracting subtle features. The present invention performs weighted fusion of the edge-point maximum correlation graph convolution and the edge-point differential graph convolution to further improve the ability of the model to extract fraud features.

[0050] The edge-point differential graph convolution that focuses on extracting subtle features subtracts the vertex features related to each hyperedge from the hyperedge features to obtain the difference, multiplies it by the attention weight, and then concatenates it with the hyperedge features; finally, skip connection is used to add it to the hyperedge features. The specific calculation formula is as follows:

[0051] ;

[0052] In the formula, represents the i-th hyperedge, represents the vertex set included in the hyperedge e i ; represents the hyperedge and the difference between the vertices connected to it, is the learnable channel attention weight, is the learnable parameter during aggregation, is the weighted edge-point difference, represents pixel multiplication in the channel dimension, and finally, the weighted edge-point differences are summed to obtain the aggregated value of the edge-point differential graph convolution, represents the edge-point differential graph convolution operation, is the concatenation function of the feature vectors. The edge-point differential graph convolution can improve the robustness of the model to noise and outliers by introducing the relative features between vertices and hyperedges, and can better aggregate hyperedge features and related vertex features.

[0053] The edge-point differential graph convolution focuses on the extraction of detailed information, while the edge-point maximum correlation graph convolution can introduce important change information in the local neighborhood. Therefore, the edge-point maximum correlation graph convolution and the edge-point differential graph convolution are weighted and fused to obtain a more comprehensive perspective and improve the robustness, flexibility, and adaptability of the model. The calculation formula of the improved edge-point differential vertex convolution operation is as follows:

[0054] ;

[0055] In the formula, is the proportionality factor that controls the influence weights of the edge-point maximum correlation graph convolution and the edge-point differential graph convolution during the update process. represents the edge-point maximum correlation graph convolution calculating the maximum difference between each vertex related to the hyperedge and the hyperedge. is the updated hyperedge feature.

[0056] The hyperedge convolution aggregates the hyperedge features into new vertex feature embeddings. As Figure 2 shown, the self-attention-based hyperedge convolution uses the self-attention mechanism in the hyperedge convolution, generates the weight scores of each hyperedge using a multi-layer perceptron (MLP), and the output new vertex features are the weighted sum of the input hyperedge features; the process formula is as follows:

[0057] ;

[0058] ;

[0059] where \(|\text{Adj}(v)|\) represents the size of the adjacent hyperedge set, represents the updated hyperedge feature, \(X\) v represents the new vertex feature, is the weight parameter matrix, \(b\) is the bias parameter matrix, and \(w\) is the weight of the hyperedge, is the \(i\)-th adjacent hyperedge, is the weight corresponding to the \(i\)-th adjacent hyperedge.

[0060] Take the features after edge-point differential vertex convolution and hyperedge convolution as the new vertex features and input them into the K-nearest neighbor and fuzzy C-clustering algorithms to continuously update the hypergraph structure; finally, combine the asymmetric triplet loss and the soft-label cross-entropy loss, and continuously backpropagate to adjust the parameters to minimize the loss.

[0061] For the negative example samples of the triplet, select samples with a moderate distance from the anchor point. They are not the negative samples closest to the anchor point, but there is also a certain distance. This can provide relatively balanced training samples, making the training both effective and not prone to overfitting.

[0062] Asymmetric triplet loss:

[0063] ;

[0064] where the anchor and the positive example have the same label, while the anchor and the negative example have different labels. is a predefined margin. The purpose is to simultaneously meet the following three requirements: separate fraudulant faces of different modalities in the feature space; aggregate all live faces of different modalities; separate fraudulant faces from all live faces.

[0065] Soft-label cross-entropy loss:

[0066] ;

[0067] where \(N\) is the number of samples, \(C\) is the number of classes, represents the probability that sample \(i\) is predicted as class \(c\), represents the true probability that sample \(i\) belongs to class \(c\).

[0068] The total loss is:

[0069] ;

[0070] Another embodiment of the present invention provides a system for implementing a face liveness detection method based on edge-point differential hypergraph convolution, including:

[0071] An image segmentation module for dividing an input face image into non-overlapping image patches;

[0072] A feature extraction module for extracting features from the image patches;

[0073] A hypergraph construction module that uses the K-nearest neighbor and fuzzy C-means clustering algorithms to construct a hypergraph structure based on the distances between features;

[0074] A hypergraph update module with built-in edge-point differential hypergraph convolution for aggregating and updating hyperedge and vertex features to obtain new vertex features;

[0075] A classifier for determining the category to which the face image belongs.

[0076] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments as well as all changes and modifications that fall within the scope of the present application.

[0077] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

Claims

1. A method for face liveness detection based on edge point differential hypergraph convolution, characterized in that: Here are the steps: S1: Divide the input face image into non-overlapping image blocks, extract features from the image blocks through the convolution layer; then use the K nearest neighbor and fuzzy C clustering algorithms to build a hypergraph structure based on the distance between features; Use the K nearest neighbor algorithm to take each vertex as the center and use the Euclidean distance of vertex features to generate a hyperedge for each vertex v; Use the fuzzy C clustering algorithm to cluster vertices globally. For each vertex v, find S-1 clusters whose cluster centers are closest to the vertex v, thereby constructing S-1 hyperedges for each vertex. Add one hyperedge constructed by the K nearest neighbor algorithm to form S hyperedge sets for each vertex. At the same time, construct a related vertex set for each hyperedge, thereby constructing the hyperedge set of the vertex and the vertex set of the hyperedge. The symbol Con(e) represents the vertex set contained in the hyperedge e, and the symbol Adj(v) represents the hyperedge set consisting of all hyperedges containing vertex v. The formula is as follows: ； ； Where n and s are the number of vertices in hyperedge e and the number of hyperedges containing vertex v, respectively. represents the nth vertex, represents the sth hyperedge; S2: The obtained hypergraph structure is sent to the edge-point differential hypergraph convolution, and information flows in the form of vertex-hyperedge-vertex, and the hyperedge and vertex features are aggregated and updated to obtain new vertex features; The edge point differential hypergraph convolution includes edge point differential vertex convolution and self-attention based hyperedge convolution, which aggregates vertex features to hyperedges through edge point differential vertex convolution, and then aggregates adjacent hyperedge features to vertices through self-attention based hyperedge convolution to obtain new vertex features; The edge point differential vertex convolution consists of the edge point maximum correlation graph convolution and the edge point differential graph convolution; the edge point maximum correlation graph convolution selects the maximum difference between the hyperedge and the connected vertices to update the hyperedge; The edge point differential graph convolution is to subtract the vertex features related to the hyperedge from the hyperedge features to get the difference, multiply it by the attention weight, and then concatenate it with the hyperedge features; finally, use a skip connection to add it to the hyperedge features; S3: The obtained new vertex features are sent to the asymmetric triplet loss function for optimization, which further promotes compactness within the class and separation between classes in the feature space; S4: Send the optimized vertex features to the classifier to obtain the category probability and determine the category to which the face image belongs.

2. The method for face liveness detection based on edge point differential hypergraph convolution according to claim 1, characterized in that: In step S1, a face image of size H × W is first evenly divided into m image blocks, and each image block is converted into a feature vector to obtain a data matrix X = [x1, x2, ..., x i , …, x m ], x i is the feature vector of the i-th image block, i = 1, 2, …, m; After that, a learnable position code is initialized, and the dimension of the position code is the same as the dimension of the feature vector of the image block; the position code is added to the feature vector to introduce the spatial position information of the feature; then the K nearest neighbor and fuzzy C clustering algorithms are used to construct a hypergraph, and the nearest neighbors are found according to the Euclidean distance between the feature vectors.

3. The method for face liveness detection based on edge point differential hypergraph convolution according to claim 1, characterized in that: The calculation formula of edge point differential graph convolution is as follows: ； In the formula, represents the i-th hyperedge, Represents the hyperedge e i the set of vertices included; Represents a hyperedge The vertices connected to it The difference, is the learnable channel attention weight, is a learnable parameter during aggregation, is the weighted edge point difference, It means to perform pixel multiplication in the channel dimension, and finally sum the weighted edge point difference values ​​to obtain the aggregate value of the edge point difference graph convolution. represents the edge point differential graph convolution operation, is the concatenation function of the eigenvectors.

4. The method for face liveness detection based on edge point differential hypergraph convolution according to claim 3 is characterized in that: The calculation formula of the edge point differential vertex convolution operation is as follows: ； In the formula, It is the proportional factor that controls the weights of the maximum correlation graph convolution of edge points and the differential graph convolution of edge points during the update process; Represents the maximum correlation graph convolution of the edge point to calculate each vertex related to the hyperedge With super edge The maximum difference of is the concatenation function of the eigenvectors; is the updated hyperedge feature.

5. The method for face liveness detection based on edge point differential hypergraph convolution according to claim 1, characterized in that: The self-attention based hyperedge convolution uses the self-attention mechanism in the hyperedge convolution, and uses a multi-layer perceptron to generate the weight score of each hyperedge. The output new vertex feature is the weighted sum of the input hyperedge features.

6. The method for face liveness detection based on edge point differential hypergraph convolution according to claim 1, characterized in that: The features obtained through edge point differential vertex convolution and hyper-edge convolution are input into the K nearest neighbor and fuzzy C clustering algorithms as new vertex features to continuously update the hypergraph structure. Finally, asymmetric triplet loss and soft label cross entropy loss are combined to continuously back-propagate and adjust parameters to minimize the loss.

7. A system for implementing the face liveness detection method according to any one of claims 1 to 6, characterized in that: include: An image segmentation module, used to divide the input face image into non-overlapping image blocks; A feature extraction module, used to extract features from image blocks; The hypergraph construction module uses K-nearest neighbor and fuzzy C clustering algorithms to build a hypergraph structure based on the distance between features; Hypergraph update module, with built-in edge-point differential hypergraph convolution, is used to aggregate and update hyperedge and vertex features to obtain new vertex features; A classifier is used to determine the category to which a face image belongs.

Images