Face living body detection method and system based on edge point difference hypergraph convolution

Abstract

The invention belongs to the technical field of face recognition, and relates to a face in-vivo detection method and system based on edge point difference hypergraph convolution, and the method comprises the steps: dividing an input face image into non-overlapping image blocks, and extracting features of the image blocks through a convolution layer; constructing a hypergraph structure according to the distance between the features by using K nearest neighbor and a fuzzy C clustering algorithm; sending the obtained hypergraph structure into edge point difference hypergraph convolution, and aggregating and updating hyperedge and vertex features to obtain new vertex features; and then optimization is carried out through an asymmetric triple loss function, the optimized vertex features are sent to a classifier, the category probability is obtained, and the category to which the face image belongs is determined. According to the method, intricate correlation existing in the image is captured based on edge point difference hypergraph convolution, the calculation complexity of the hypergraph is reduced in combination with fuzzy C clustering, and human face in-vivo detection with high detection rate and low error rate is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of face recognition, and particularly relates to a face liveness detection method and system based on edge-point differential hypergraph convolution. Background Art

[0002] To detect and defend against attacks, existing face liveness detection (FAS) solutions include domain adversarial generalization techniques, disentanglement techniques, GNN models, and so on. The emergence of domain adversarial generalization techniques is to alleviate the domain shift phenomenon caused by the different data distributions between the training domain and the test domain, and can improve the generalization of the model. The role of disentanglement techniques is to separate fraud clues from factors such as the environment and facial features of the face to improve the robustness of the model. The GNN model is deployed based on paired simple graphs. Although this graph construction method can provide additional discriminant information between vertices and improve the robustness of the model, it is limited to simple paired relationships, resulting in redundant edges and unnecessary memory and computational overhead. Secondly, a basic limitation of simple graphs is that they uniquely connect two vertices, thus uniquely accommodating paired relationships. When it comes to modeling the inherent high-order relationships in images, this deficiency is obvious. Summary of the Invention

[0003] The purpose of the present invention is to provide a face liveness detection method and system based on edge-point differential hypergraph convolution, which uses hypergraph technology based on edge-point differential hypergraph convolution to capture the intricate correlations existing in images, and at the same time combines improved fuzzy C clustering to reduce the computational complexity of the hypergraph. The hypergraph transcends the limitations of paired relationships and effectively integrates complex high-order relationships in a unified hypergraph to achieve face liveness detection with high detection rate and low error rate.

[0004] The present invention is realized through the following technical solutions: A face liveness detection method based on edge-point differential hypergraph convolution, the steps are as follows: S1: Divide the input face image into non-overlapping image patches, extract features from the image patches through a convolutional layer; then use the K-nearest neighbor and fuzzy C clustering algorithms to construct a hypergraph structure according to the distances between the features; S2: Feed the obtained hypergraph structure into edge-point differential hypergraph convolution to flow information in the vertex-hyperedge-vertex manner, aggregate and update the hyperedge and vertex features, and obtain new vertex features. The edge-point differential hypergraph convolution includes edge-point differential vertex convolution and self-attention-based hyperedge convolution. The vertex features are aggregated to the hyperedges through edge-point differential vertex convolution, and then the adjacent hyperedge features are aggregated to the vertices through self-attention-based hyperedge convolution to obtain new vertex features. The edge-point differential vertex convolution consists of edge-point maximum correlation graph convolution and edge-point differential graph convolution. The edge-point maximum correlation graph convolution selects the maximum difference between the hyperedge and the connected vertices to update the hyperedge. The edge-point differential graph convolution subtracts the vertex features related to each hyperedge from the hyperedge features to obtain the difference, multiplies it by the attention weight, and then concatenates it with the hyperedge features. Finally, skip connection is used and added to the hyperedge features. S3: Feed the obtained new vertex features into the asymmetric triplet loss function for optimization to further promote the compactness within the class and the separation between classes in the feature space. S4: Feed the optimized vertex features into the classifier to obtain the class probabilities and determine the class to which the face image belongs.

[0005] Specifically, in step S1, first, a face image with a size of H × W is evenly divided into m image patches. After converting each image patch into a feature vector, the data matrix X = [x 1 , x 2 , …, x i , …, x m is obtained, where x i is the feature vector of the i-th image patch, i = 1, 2, …, m. Then, a learnable position encoding is initialized, and the dimension of the position encoding is the same as that of the feature vector of the image patch. The position encoding is added to the feature vector to introduce the spatial position information of the feature. Then, the K-nearest neighbor and fuzzy C-means clustering algorithms are used to construct a hypergraph, and the nearest neighbors are found according to the Euclidean distance between the feature vectors.

[0006] Specifically, in step S1, using the K-nearest neighbor algorithm, with each vertex as the center, 1 hyperedge is generated for each vertex v using the Euclidean distance of the vertex features. Using the fuzzy C-means clustering algorithm for vertex clustering globally, for each vertex v, S - 1 clusters with the closest cluster center to vertex v are found, so as to construct S - 1 hyperedges for each vertex. Adding the 1 hyperedge formed by the K-nearest neighbor algorithm, S hyperedge sets are formed for each vertex. At the same time, a related vertex set is constructed for each hyperedge, thus constructing the hyperedge set of the vertices and the vertex set of the hyperedges. Use the symbol Con( e ) to represent the vertex set contained in the hyperedge e, and use the symbol Adj( v ) to represent the hyperedge set composed of all hyperedges containing the vertex v. The formula is as follows: ; ; where n and s are the number of vertices in the hyperedge e and the number of hyperedges containing the vertex v, respectively. represents the nth vertex, represents the sth hyperedge.

[0007] Specifically, the calculation formula of edge-point differential graph convolution is as follows: ; In the formula, represents the ith hyperedge, represents the hyperedge e i contains the vertex set; represents the hyperedge and its connected vertex difference, is the learnable channel attention weight, is the learnable parameter during aggregation, is the weighted edge-point difference, represents pixel multiplication in the channel dimension, and finally, the weighted edge-point differences are summed to obtain the aggregated value of the edge-point differential graph convolution. represents the edge-point differential graph convolution operation, is the concatenation function of the feature vectors.

[0008] Specifically, the calculation formula of the edge-point differential vertex convolution operation is as follows: ; In the formula, is the scaling factor that controls the influence weights of the edge-point maximum correlation graph convolution and the edge-point differential graph convolution during the update process. represents the edge-point maximum correlation graph convolution calculating the maximum difference between each vertex related to the hyperedge and the hyperedge is the concatenation function of the feature vectors; is the updated hyperedge feature.

[0009] Further preferably, the hyperedge convolution based on self-attention uses the self-attention mechanism in the hyperedge convolution, generates the weight scores of each hyperedge using a multi-layer perceptron, and the output new vertex features are the weighted sum of the input hyperedge features.

[0010] Further preferably, the features after the edge-point differential vertex convolution and the hyperedge convolution are used as the new vertex features and input into the K-nearest neighbor and fuzzy C-clustering algorithms to continuously update the hypergraph structure; finally, combined with the asymmetric triplet loss and the soft label cross-entropy loss, continuously backpropagate and adjust the parameters to minimize the loss.

[0011] The present invention also provides a system for implementing a face liveness detection method based on edge point differential hypergraph convolution, including: An image segmentation module, configured to divide an input face image into non-overlapping image patches; A feature extraction module, configured to extract features from the image patches; A hypergraph construction module, using the K-nearest neighbor and fuzzy C-means clustering algorithms, to construct a hypergraph structure according to the distances between features; A hypergraph update module, built-in with edge point differential hypergraph convolution, for aggregating and updating hyperedge and vertex features to obtain new vertex features; A classifier, configured to determine the category to which the face image belongs.

[0012] The present invention has the following advantages: (1) In face liveness detection, a hypergraph can simultaneously capture complex interactions between multiple features, such as information on illumination changes, facial texture details, reflections, and shadows. This many-to-many relationship better depicts the subtle differences between real faces and fake faces. The hypergraph structure connects multiple vertices through hyperedges, enabling large-scale information propagation and aggregation in a small number of steps. For face liveness detection, this means that the model can more quickly capture the interrelated information of different parts, improving the detection accuracy and efficiency. By utilizing the powerful complex data representation ability of the hypergraph, the ability to extract subtle features of the liveness model is improved, thereby enhancing the detection performance of the model.

[0013] (2) By adopting a dynamic hypergraph model to learn and generate a hypergraph, the extracted features can retain the original high-dimensional features of the face as much as possible, and through edge point differential vertex convolution, the problem of excessive information abstraction is alleviated, greatly enhancing the accuracy of the face liveness detection model.

[0014] (3) When the face test data set is input into the hypergraph face liveness detection model, the use of the asymmetric triplet loss method makes the face features generate obvious class boundaries in the feature space.

[0015] (4) The improved fuzzy C-means clustering algorithm can reduce the computational overhead caused by partial repeated calculations. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the method of the present invention.

[0017] Figure 2 It is a schematic diagram of hyperedge convolution based on self-attention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The present invention will be further described in detail below with reference to the accompanying drawings.

[0019] Refer to Figure 1, A face liveness detection method based on edge-point differential hypergraph convolution, the steps are as follows: S1: Divide the input face image into non-overlapping image patches, extract features from the image patches through a convolutional layer; then use the K-nearest neighbor and fuzzy C-means clustering algorithms to construct a hypergraph structure based on the distance between features; S2: Feed the obtained hypergraph structure into the edge-point differential hypergraph convolution to flow information in the vertex-hyperedge-vertex manner, aggregate and update the hyperedge and vertex features to obtain new vertex features; S3: Feed the obtained new vertex features into the asymmetric triplet loss function for optimization to further promote intra-class compactness and inter-class separation in the feature space; S4: Feed the optimized vertex features into a classifier to obtain class probabilities and determine the class to which the face image belongs.

[0020] In step S1, first, a face image of size H × W is evenly divided into m image patches. After converting each image patch into a feature vector, a data matrix X = [x 1 , x 2 , …, x i , …, x m is obtained, where x i is the feature vector of the i-th image patch, i = 1, 2, …, m. Then, a learnable position encoding is initialized, and the dimension of the position encoding is the same as that of the feature vector of the image patch. Add the position encoding to the feature vector to introduce the spatial position information of the feature. Then use the K-nearest neighbor and fuzzy C-means clustering algorithms to construct a hypergraph, and find the nearest neighbors according to the Euclidean distance between the feature vectors.

[0021] In the present invention, the K-nearest neighbor algorithm is used to generate 1 hyperedge for each vertex v with the Euclidean distance of the vertex features as the center, and the generated hyperedges have obvious locality; for the improvement of the fuzzy C-means clustering algorithm: starting from the second layer, abandon the scheme of initializing the clustering center at the beginning of each layer, and instead replace it with the method of using the clustering center of the previous layer in each layer. In this way, during the process of clustering convergence, some repeated calculations can be reduced. The improved fuzzy C-means clustering algorithm can perform vertex clustering globally. For each vertex v, S - 1 clusters with the closest clustering center to vertex v can be found, so as to construct S - 1 hyperedges for each vertex. Together with the 1 hyperedge formed by the K-nearest neighbor algorithm, S hyperedge sets can be formed for each vertex. At the same time, construct the relevant vertex set for each hyperedge, so as to construct the hyperedge set of vertices and the vertex set of hyperedges.

[0022] Use the symbol Con(e) to represent the vertex set contained in the hyperedge e, and use the symbol Adj(v) to represent the hyperedge set composed of all hyperedges containing the vertex v. The formula is as follows: ; ; where \(n\) and \(s\) are the number of vertices in the hyperedge \(e\) and the number of hyperedges containing the vertex \(v\) respectively, represents the \(n\)-th vertex, represents the \(s\)-th hyperedge.

[0023] The edge-point differential hypergraph convolution of the present invention includes edge-point differential vertex convolution and self-attention-based hyperedge convolution. The vertex features are aggregated into hyperedges through edge-point differential vertex convolution, and then the adjacent hyperedge features are aggregated into vertices through self-attention-based hyperedge convolution to obtain new vertex feature embeddings.

[0024] Among them, the edge-point differential vertex convolution consists of edge-point maximum correlation graph convolution and edge-point differential graph convolution. The edge-point maximum correlation graph convolution selects the maximum difference between the hyperedge and the connected vertices to update the hyperedge. This method helps to introduce important change information in the local neighborhood. However, it will ignore some subtle features. The edge-point differential graph convolution focuses on extracting subtle features. The present invention performs weighted fusion of the edge-point maximum correlation graph convolution and the edge-point differential graph convolution to further improve the ability of the model to extract fraud features.

[0025] The edge-point differential graph convolution that focuses on extracting subtle features subtracts the vertex features related to each hyperedge from the hyperedge features to obtain the difference, multiplies it by the attention weight, and then concatenates it with the hyperedge features; finally, skip connection is used to add it to the hyperedge features. The specific calculation formula is as follows: ; In the formula, represents the \(i\)-th hyperedge, represents the hyperedge \(e\) i contains the vertex set; represents the hyperedge and the difference between the vertices connected to it, is the learnable channel attention weight, is the learnable parameter during aggregation, is the weighted edge-point difference, represents pixel multiplication in the channel dimension. Finally, the weighted edge-point differences are summed to obtain the aggregation value of the edge-point differential graph convolution, represents the edge-point differential graph convolution operation, is the concatenation function of feature vectors. The edge-point differential graph convolution can improve the robustness of the model to noise and outliers by introducing the relative features between vertices and hyperedges, and can better aggregate hyperedge features and related vertex features.

[0026] Edge - point differential graph convolution focuses on the extraction of detailed information, while edge - point maximum - correlation graph convolution can introduce important change information in the local neighborhood. Therefore, edge - point maximum - correlation graph convolution and edge - point differential graph convolution are weighted and fused in order to obtain a more comprehensive perspective and improve the robustness, flexibility, and adaptability of the model. The calculation formula for the improved edge - point differential vertex convolution operation is as follows: ; In the formula, is the proportionality factor that controls the influence weights of edge - point maximum - correlation graph convolution and edge - point differential graph convolution during the update process. represents the edge - point maximum - correlation graph convolution calculating the maximum difference between each vertex associated with the hyper - edge and the hyper - edge. is the updated hyper - edge feature.

[0027] Hyper - edge convolution aggregates the hyper - edge features into new vertex feature embeddings. As shown in Figure 2 , the self - attention - based hyper - edge convolution uses the self - attention mechanism in hyper - edge convolution, generates the weight scores for each hyper - edge using a multi - layer perceptron (MLP), and the output new vertex feature is the weighted sum of the input hyper - edge features. The process formula is as follows: ; ; In the formula, |Adj(v)| represents the size of the adjacent hyper - edge set, represents the updated hyper - edge feature, X v represents the new vertex feature, is the weight parameter matrix, b is the bias parameter matrix, w is the weight of the hyper - edge, is the i - th adjacent hyper - edge, is the weight corresponding to the i - th adjacent hyper - edge.

[0028] Take the features after edge - point differential vertex convolution and hyper - edge convolution as the new vertex features and input them into the K - nearest neighbor and fuzzy C - clustering algorithms to continuously update the hyper - graph structure; finally, combine the asymmetric triplet loss and the soft - label cross - entropy loss, and continuously back - propagate to adjust the parameters to minimize the loss.

[0029] For the negative example samples of the triplet, samples with a moderate distance from the anchor point are selected. They are not the negative samples closest to the anchor point but still have a certain distance. This can provide relatively balanced training samples, making the training both effective and not prone to overfitting.

[0030] Asymmetric triplet loss: ; Among them, the anchor and the positive example have the same label, while the anchor points and negative examples have different labels. is a predefined boundary. The purpose is to simultaneously meet the following three requirements: separate fraudulent faces of different modalities in the feature space; aggregate all live faces of different modalities; and separate fraudulent faces from all live faces.

[0031] Soft label cross-entropy loss: ; where N is the number of samples, C is the number of classes, represents the probability that sample i is predicted as class c, represents the true probability that sample i belongs to class c.

[0032] The total loss is: ; Another embodiment of the present invention provides a system for implementing a face liveness detection method based on edge point differential hypergraph convolution, including: An image segmentation module for dividing the input face image into non-overlapping image patches; A feature extraction module for extracting features from the image patches; A hypergraph construction module that uses the K-nearest neighbor and fuzzy C-means clustering algorithms to construct a hypergraph structure based on the distance between features; A hypergraph update module with built-in edge point differential hypergraph convolution for aggregating and updating hyperedge and vertex features to obtain new vertex features; A classifier for determining the class to which the face image belongs.

[0033] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the present application.

[0034] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A method for face liveness detection based on edge point differential hypergraph convolution, characterized in that: Here are the steps: S1: Divide the input face image into non-overlapping image blocks, extract features from the image blocks through the convolution layer; then use the K nearest neighbor and fuzzy C clustering algorithms to build a hypergraph structure based on the distance between features; S2: The obtained hypergraph structure is sent to the edge-point differential hypergraph convolution, and information flows in the form of vertex-hyperedge-vertex, and the hyperedge and vertex features are aggregated and updated to obtain new vertex features; The edge point differential hypergraph convolution includes edge point differential vertex convolution and self-attention based hyperedge convolution, which aggregates vertex features to hyperedges through edge point differential vertex convolution, and then aggregates adjacent hyperedge features to vertices through self-attention based hyperedge convolution to obtain new vertex features; The edge point differential vertex convolution consists of the edge point maximum correlation graph convolution and the edge point differential graph convolution; the edge point maximum correlation graph convolution selects the maximum difference between the hyperedge and the connected vertices to update the hyperedge; The edge point differential graph convolution is to subtract the vertex features related to the hyperedge from the hyperedge features to get the difference, multiply it by the attention weight, and then concatenate it with the hyperedge features; finally, use a skip connection to add it to the hyperedge features; S3: The obtained new vertex features are sent to the asymmetric triplet loss function for optimization, which further promotes compactness within the class and separation between classes in the feature space; S4: Send the optimized vertex features to the classifier to obtain the category probability and determine the category to which the face image belongs.

2. The method for face liveness detection based on edge point differential hypergraph convolution according to claim 1, characterized in that: In step S1, a face image of size H × W is first evenly divided into m image blocks, and each image block is converted into a feature vector to obtain a data matrix X = [x1, x2, ..., x i , …, x m ], x i is the feature vector of the i-th image block, i = 1, 2, …, m; After that, a learnable position code is initialized, and the dimension of the position code is the same as the dimension of the feature vector of the image block; the position code is added to the feature vector to introduce the spatial position information of the feature; then the K nearest neighbor and fuzzy C clustering algorithms are used to construct a hypergraph, and the nearest neighbors are found according to the Euclidean distance between the feature vectors.

3. The method for face liveness detection based on edge point differential hypergraph convolution according to claim 2 is characterized in that: In step S1, the K nearest neighbor algorithm is used to generate a hyperedge for each vertex v using the Euclidean distance of vertex features with each vertex as the center; the fuzzy C clustering algorithm is used to perform vertex clustering globally, and for each vertex v, S-1 clusters whose cluster centers are closest to the vertex v are found, thereby constructing S-1 hyperedges for each vertex, and adding one hyperedge constructed by the K nearest neighbor algorithm, forming S hyperedge sets for each vertex; at the same time, a related vertex set is constructed for each hyperedge, thereby constructing a hyperedge set of the vertex and a vertex set of the hyperedge; The symbol Con(e) represents the vertex set contained in the hyperedge e, and the symbol Adj(v) represents the hyperedge set consisting of all hyperedges containing vertex v. The formula is as follows: ； ； Where n and s are the number of vertices in hyperedge e and the number of hyperedges containing vertex v, respectively. represents the nth vertex, represents the sth hyperedge.

4. The method for face liveness detection based on edge point differential hypergraph convolution according to claim 3 is characterized in that: The calculation formula of edge point differential graph convolution is as follows: ； In the formula, represents the i-th hyperedge, Represents the hyperedge e i the set of vertices included; Represents a hyperedge The vertices connected to it The difference, is the learnable channel attention weight, is a learnable parameter during aggregation, is the weighted edge point difference, It means to perform pixel multiplication in the channel dimension, and finally sum the weighted edge point difference values ​​to obtain the aggregate value of the edge point difference graph convolution. represents the edge point differential graph convolution operation, is the concatenation function of the eigenvectors.

5. The method for face liveness detection based on edge point differential hypergraph convolution according to claim 4 is characterized in that: The calculation formula of the edge point differential vertex convolution operation is as follows: ； In the formula, It is the proportional factor that controls the weights of the maximum correlation graph convolution of edge points and the differential graph convolution of edge points during the update process; Represents the maximum correlation graph convolution of the edge point to calculate each vertex related to the hyperedge With super edge The maximum difference of is the concatenation function of the eigenvectors; is the updated hyperedge feature.

6. The method for face liveness detection based on edge point differential hypergraph convolution according to claim 1, characterized in that: The self-attention based hyperedge convolution uses the self-attention mechanism in the hyperedge convolution, and uses a multi-layer perceptron to generate the weight score of each hyperedge. The output new vertex feature is the weighted sum of the input hyperedge features.

7. The method for face liveness detection based on edge point differential hypergraph convolution according to claim 1, characterized in that: The features obtained through edge point differential vertex convolution and hyper-edge convolution are input into the K nearest neighbor and fuzzy C clustering algorithms as new vertex features to continuously update the hypergraph structure. Finally, asymmetric triplet loss and soft label cross entropy loss are combined to continuously back-propagate and adjust parameters to minimize the loss.

8. A system for implementing the face liveness detection method according to any one of claims 1 to 7, characterized in that: include: An image segmentation module, used to divide the input face image into non-overlapping image blocks; A feature extraction module, used to extract features from image blocks; The hypergraph construction module uses K-nearest neighbor and fuzzy C clustering algorithms to build a hypergraph structure based on the distance between features; Hypergraph update module, with built-in edge-point differential hypergraph convolution, is used to aggregate and update hyperedge and vertex features to obtain new vertex features; A classifier is used to determine the category to which a face image belongs.

Images