A face forgery detection method and system based on spatio-temporal hypergraph

Abstract

The application discloses the technical field of image processing and relates to a face forgery detection method and system based on a hypergraph of a hyperbolic space-time, and aims to solve the problem that the prior art is difficult to capture the internal feature connection of a face structure. The method comprises the following steps: inputting a face video to be detected for authenticity into a trained face forgery detection model; performing feature extraction on the face video to be detected for authenticity through a face feature partition extraction module to obtain face local features; constructing a space-time hypergraph through a space-time node hypergraph construction module according to the face local features; performing semantic classification feature extraction on the space-time hypergraph through a hyperbolic hypergraph learning module to obtain classification features; and inputting the classification features into a classifier to obtain a detection result of true or false. The application can realize face forgery detection with high accuracy and strong generalization, and can be widely applied to the fields of security and finance, thereby providing reliable protection for related face recognition applications.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a face forgery detection method and system based on a spatio-temporal hyperbolic hypergraph. BACKGROUND

[0002] Traditional face video forgery can only achieve simple face replacement effect, and the result is not only rough but also easy to be recognized. Recently, with the rapid development of deep learning in the field of computer vision, face forgery technology has made certain breakthroughs. Advanced face deep forgery technologies such as generative adversarial networks and diffusion models can not only achieve high-precision face swapping but also accurately edit face attributes and realistically synthesize faces, so that a large number of false face images and videos have reached the level of fake to real. Although these deep face forgery technologies can enrich the form of content creation and open up the space for content creation, they also pose potential threats to society and the general public.

[0003] Therefore, face forgery detection technology emerges as the times require, aiming to accurately determine the authenticity of face images or videos. Early detection methods have certain effect on simple forgery samples, but they are inadequate when faced with increasingly updated deep forgery technologies, and detection methods with low precision often lead to serious security problems. After the rise of deep learning, face forgery detection technology has been improved to some extent, but due to the lack of sufficient mining of spatio-temporal complex structure and semantic information of face video, the detection accuracy and generalization ability still have shortcomings, which makes it difficult to meet the actual needs.

[0004] Although some existing face forgery detection models have begun to consider the spatio-temporal complex structure and semantic information of face video, most of them perform feature learning and processing in Euclidean space. However, Euclidean space often faces great challenges when representing data with complex hierarchical structure and highly sparse characteristics, making it difficult to capture the intrinsic feature relationship of face structure. In practical applications, face video often contains not only pixel-level detail information but also obvious hierarchical structure, such as spatial hierarchical relationship between different facial organs and temporal and hierarchical relationship between different frames in the video. Therefore, the problem of how to effectively mine face structured information and fully utilize its hierarchical relationship for feature extraction and discrimination remains unsolved. SUMMARY

[0005] The present application aims to overcome the shortcomings of the prior art and provide a face forgery detection method and system based on a spatio-temporal hyperbolic hypergraph, which aims to model the intra-frame structural nature, inter-frame temporal nature, and cross-frame similarity of face video in a spatio-temporal hypergraph manner, and introduce a hyperbolic hypergraph network to focus on feature hierarchical relationship learning, thereby achieving face forgery detection with high accuracy and strong generalization.

[0006] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution:

[0007] In a first aspect, the present invention provides a face forgery detection method based on spatiotemporal hyperbolic hypergraph, comprising:

[0008] The authenticity of the face video to be detected is input into the trained face forgery detection model: the face feature partitioning extraction module extracts features from the face video to obtain local face features; based on the local face features, a spatiotemporal hypergraph is constructed using a spatiotemporal node hypergraph construction module; semantic classification features are extracted from the spatiotemporal hypergraph using a hyperbolic hypergraph learning module to obtain classification features; the classification features are input into a classifier to obtain the detection result as true or false.

[0009] Optionally, the face feature partitioning extraction module includes a face region extraction module and a local feature extraction module;

[0010] The face region extraction module includes a face key point detector and a face region segmenter; the face key point detector is based on the authenticity of the face video to be detected. Locate each frame of the image. Key points The facial region segmenter is based on key points. Classify faces into Facial area ;in, , , , This indicates the number of frames in the face video to be tested for authenticity. Indicates the frame number. Indicates the first Frame image, Indicates the first The first frame of the image One key point, Indicates the first The first frame of the image One facial area;

[0011] The local feature extraction module extracts features based on each facial region. Using pre-trained convolutional neural networks Feature extraction is performed to obtain the facial region. eigenvectors Output the local facial features ,in, , , Indicates the first In the frame image A set of feature vectors for each facial region .

[0012] Optionally, the spatio-temporal node hypergraph construction module takes the local facial features as nodes, constructs hyperedges by using a rule-based hyperedge construction method and a similarity-based hyperedge construction method, and obtains a spatio-temporal hypergraph.

[0013] The expression of the spatio-temporal hypergraph is:

[0014] ;

[0015] wherein, denotes a node set, , denotes an i-th node, denotes a frame number of a face video to be detected for authenticity, denotes a total number of face regions, denotes a hyperedge set, , denotes a time-dimension hyperedge, denotes a region-dimension hyperedge, denotes a similarity hyperedge.

[0016] Optionally, the rule-based hyperedge construction method establishes a connection relationship between nodes in the time dimension and the region dimension, and obtains the time-dimension hyperedge and the region-dimension hyperedge.

[0017] The time-dimension hyperedge is constructed for nodes of different frames with the same face region, and the expression of the time-dimension hyperedge is:

[0018] ;

[0019] wherein, denotes a feature vector of an i-th face region in an i-th frame image, denotes a corresponding node, denotes a feature vector of an i-th face region in an i-th frame image, denotes a corresponding node; The region-dimension hyperedge is constructed for different face regions of a same frame node, and the expression of the region-dimension hyperedge is:

[0020]

[0021] ;

[0022] wherein, denotes a feature vector of an i-th face region in an i-th frame image, denotes a corresponding node, denotes a feature vector of an i-th face region in an i-th frame image, ​​​​​Corresponding node, representing the frame image feature vector of the corresponding node;

[0023] The similarity-based hyperedge construction method selects the most similar nodes to the node through a node similarity function to obtain a similarity hyperedge;

[0024] The expression of the similarity hyperedge is:

[0025] ;

[0026] wherein, representing the node, representing the set of the most similar nodes to the node, representing the similar node of the , representing the similar node of the , representing the similar node of the , representing the similar node of the .

[0027] Optionally, the hyperbolic hypergraph learning module includes a hyperbolic exponential mapping layer, a hyperbolic learning module, and a hyperbolic logarithmic mapping layer.

[0028] The hyperbolic hypergraph learning module performs semantic classification feature extraction on the spatio-temporal hypergraph, including:

[0029] The hyperbolic exponential mapping layer is used to map the feature vector of the spatio-temporal hypergraph from Euclidean space to hyperbolic space, obtaining a mapped feature.

[0030] The hyperbolic learning module is used to perform semantic classification feature extraction according to the mapped feature, obtaining a hyperbolic space classification feature.

[0031] The hyperbolic logarithmic mapping layer is used to map the hyperbolic space classification feature back to Euclidean space, obtaining a classification feature.

[0032] Optionally, the expression of the hyperbolic exponential mapping layer is as follows:

[0033] ;

[0034] wherein, representing the hyperbolic curvature, representing the origin,​ represents an exponential mapping function with as origin and as hyperbolic curvature, represents a characteristic vector of a hyperbolic space classification feature in Euclidean space;

[0035] The expression of the hyperbolic logarithmic mapping layer is as follows:

[0036] ;

[0037] wherein, represents a hyperbolic space classification feature, represents a logarithmic mapping function with as origin and as hyperbolic curvature.

[0038] Optionally, the hyperbolic learning module at least includes a set of hyperbolic feature transformation layers, hyperbolic neighborhood aggregation layers and nonlinear activation layers connected in series;

[0039] The hyperbolic learning module is used for semantic classification feature extraction, comprising:

[0040] The hyperbolic feature transformation layer is used for linear transformation of the mapped feature to obtain linear transformation features;

[0041] The hyperbolic neighborhood aggregation layer is used for weighted aggregation according to the linear transformation features and the linear transformation features of the neighborhood nodes of the corresponding nodes of the linear transformation features to obtain aggregated features;

[0042] The nonlinear activation layer is used for nonlinear transformation of the aggregated features to obtain hyperbolic space classification features.

[0043] Optionally, the expression of the hyperbolic feature transformation layer is as follows:

[0044] ;

[0045] wherein, represents a characteristic vector of the th node in the th hyperbolic feature transformation layer, represents a characteristic vector of the th node in the th hyperbolic feature transformation layer, represents a weight matrix of the th hyperbolic feature transformation layer, represents a bias term of the th hyperbolic feature transformation layer, represents Möbius scalar multiplication;

[0046] The expression for the hyperbolic neighborhood aggregation layer is as follows:

[0047] ;

[0048] in, Indicates the first The characteristics of each node express Features after aggregation Indicates hyperbolic curvature. Indicates the first The node and the first Hyperedges between nodes Indicates the superedge The weight, Indicates the first The feature set of the neighboring nodes of a node. Indicates With the origin It is a logarithmic mapping function of hyperbolic curvature. Indicates With the origin It is an exponential mapping function of hyperbolic curvature;

[0049] The expression for the nonlinear activation layer is as follows:

[0050] ;

[0051] in, This represents the input to the nonlinear activation layer. This represents the output of the nonlinear activation layer. Indicates the first The hyperbolic curvature of the nonlinear activation layer Indicates the first The hyperbolic curvature of the nonlinear activation layer Indicates With the origin It is an exponential mapping function of hyperbolic curvature. Indicates With the origin It is a logarithmic mapping function of hyperbolic curvature. This represents the activation function.

[0052] Optionally, the classifier includes one fully connected layer, the input size of which is the size of the classification feature, and the output size of which is 2, representing the probabilities of detecting as true and false, respectively.

[0053] The training process of the face forgery detection model includes:

[0054] Construct a training data set, the training data set includes real face video data and face video data generated by using a fake technology, all video data are equipped with corresponding labels for distinguishing real data and fake data;

[0055] According to the training data set, the cross-entropy loss of the face fake detection model is calculated;

[0056] The face fake detection model is iteratively updated and trained based on the gradient descent method to minimize the cross-entropy loss, and a trained face fake detection model is obtained;

[0057] The formula for calculating the cross-entropy loss is:

[0058] ;

[0059] Wherein, Indicates the label, Indicates the detection result, Indicates the loss between the detection result And the label .

[0060] In the second aspect, the application provides a face fake detection system based on a spatio-temporal hypergraph, comprising:

[0061] A data acquisition module is configured to acquire a face video to be detected for authenticity;

[0062] A face feature partition extraction module is configured to perform feature extraction on the face video to be detected for authenticity to obtain face local features, and obtain a detection result;

[0063] A spatio-temporal node hypergraph construction module is configured to construct a spatio-temporal hypergraph according to the face local features;

[0064] A hyperbolic hypergraph learning module is configured to perform semantic classification feature extraction on the spatio-temporal hypergraph to obtain classification features;

[0065] A detection module is configured to input the classification features into a classifier to obtain a detection result.

[0066] Compared with the prior art, the application has the following beneficial effects:

[0067] 1.The face forgery detection method based on hyperbolic hypergraph of space-time is provided, the face feature partition extraction module is used for positioning key points to divide face regions and deeply extract features, and exclusive features of each face region are fully mined; the space-time node hypergraph construction module is used for taking local feature data as nodes, combining rules and similarity to construct hyperedges, fully capturing complex high-order relationships in space-time dimensions and integrating information; and the hyperbolic hypergraph learning module is used for mapping the space-time hypergraph to a hyperbolic space for hyperbolic hypergraph convolution aggregation, using unique geometric properties of the hyperbolic space, focusing on learning of feature hierarchical relationships, and improving the feature learning capability.

[0068] 2.The face forgery detection system based on hyperbolic hypergraph of space-time is provided, the data acquisition module, the face feature partition extraction module, the space-time node hypergraph construction module, the hyperbolic hypergraph learning module and the detection module are arranged, and the face forgery detection based on the hyperbolic hypergraph of space-time is realized, the space-time consistency features of the face forgery can be more accurately extracted, higher accuracy and stronger generalization capability are achieved, and the system has practical significance and good application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0069] Figure 1 A face forgery detection method flowchart based on hyperbolic hypergraph of space-time according to an embodiment of the present application is provided.

[0070] Figure 2 A structure diagram of a face forgery detection model according to an embodiment of the present application is provided.

[0071] Figure 3 A structure diagram of a face feature partition extraction module according to an embodiment of the present application is provided.

[0072] Figure 4 A structure diagram of a space-time node hypergraph construction module according to an embodiment of the present application is provided.

[0073] Figure 5 A structure diagram of a hyperbolic hypergraph learning module according to an embodiment of the present application is provided. DETAILED DESCRIPTION

[0074] The technical scheme of the present application will be described in detail below with the help of the drawings and specific embodiments, and it should be understood that the specific features in the embodiments and the specific embodiments of the present application are detailed descriptions of the technical scheme of the present application, and are not limitations of the technical scheme of the present application, and the technical features in the embodiments and the specific embodiments can be combined with each other without conflict.

[0075] It should be noted that the term "and / or" in this article is only to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. In addition, the character " / " in this article generally represents that the associated objects before and after it are in an "or" relationship.

[0076] Embodiment one:

[0077] The embodiment of the application discloses a face forgery detection method based on a space-time hypergraph, which refers to Figure 1 and Figure 2 As shown, it specifically includes the following steps:

[0078] S1, input the face video to be detected for authenticity into a trained face forgery detection model:

[0079] S2, extract features from the face video to be detected for authenticity through a face feature partition extraction module to obtain local face features;

[0080] S3, construct a space-time hypergraph according to the local face features through a space-time node hypergraph construction module;

[0081] S4, extract semantic classification features from the space-time hypergraph through a hyperbolic hypergraph learning module to obtain classification features;

[0082] S5, input the classification features into a classifier to obtain a detection result as true or false.

[0083] Specifically, in step S1, the training process of the face forgery detection model includes:

[0084] I. Construct a training data set, which includes real face video data and face video data generated by using forgery technology, and all video data are equipped with corresponding labels for distinguishing real data and forged data; In this embodiment, data is obtained from public face forgery detection data sets such as FaceForensics++ (FF++), DeepFake Detection Challenge (DFDC), Celeb-DF (CDF) and the like; Detect the face by means of Dlib tool, align the detected face by means of affine transformation, set a fixed margin (for example, 1.3) for the face area, and then implement a cropping operation, and finally scale the face image obtained by cropping to a fixed pixel size (in this example, it is set to 256*256 pixels);

[0085] II. Input the real face video data and the face video data generated by the existing forgery technology into the pre-constructed face forgery detection model to obtain a model prediction result The label for real face video data is 0, and the label for face video data generated by existing forgery technology is 1.

[0086] 3. Based on the training dataset, calculate the cross-entropy loss of the face forgery detection model; the formula for calculating the cross-entropy loss is:

[0087] ;

[0088] in, Indicates a label, Indicates the test results. Indicates the test results and tags The losses between;

[0089] Fourth, the face forgery detection model is iteratively updated and trained using the gradient descent method to minimize the cross-entropy loss, thereby obtaining a well-trained face forgery detection model.

[0090] In step S2, refer to Figure 3 As shown, the face feature partitioning extraction module includes a face region extraction module and a local feature extraction module; the face feature partitioning extraction module divides the input face video to be detected as real or fake into regions through the face region extraction module and the local feature extraction module, and generates local face features.

[0091] The face region extraction module includes a face key point detector and a face region segmenter; the face key point detector is based on the authenticity of the face video to be detected. Locate each frame of the image. Key points The facial region segmenter is based on key points. Classify faces into Facial area ;in, , , , This indicates the number of frames in the face video to be tested for authenticity. Indicates the frame number. Indicates the first Frame image, Indicates the first The first frame of the image One key point, Indicates the first The first frame of the image One facial area;

[0092] The local feature extraction module extracts features based on each facial region. Using pre-trained convolutional neural networks feature extraction is performed to obtain a feature vector of the face region , the face local feature is output , , , denotes a feature vector set of the face region in the i-th frame image, . The local feature extraction module performs feature extraction on multiple face regions to generate a face local feature; the local feature extraction module uses a pre-trained convolutional neural network to perform feature extraction according to each face region ,

[0093] obtain a feature vector of the face region , output the face local feature , wherein, denotes a feature vector set of the face region in the i-th frame image. In step S3, as shown in FIG. 3, the spatio-temporal node hypergraph construction module takes the face local feature as a node, constructs a hyperedge by using a rule-based hyperedge construction method and a similarity-based hyperedge construction method, and obtains a spatio-temporal hypergraph; the rule-based hyperedge construction method establishes an explicit connection relationship from two dimensions of time and region; the similarity-based hyperedge construction method finds the most similar

[0094] Figure 4 nodes for the node according to the similarity of the node features and constructs a hyperedge.

[0095] The expression of the spatio-temporal hypergraph is as follows:

[0096] ;

[0097] wherein, denotes a node set, , denotes the i-th node, denotes the frame number of the face video to be detected for authenticity, denotes the total number of face regions, denotes a hyperedge set, , denotes a time-dimension hyperedge, denotes a region-dimension hyperedge, denotes a similarity hyperedge.

[0098] ​​​​​The temporal hyperedge is constructed for nodes in different frames that share the same facial region, and its expression is:

[0099] ;

[0100] in, Indicates the first The first frame of the image Feature vectors of facial regions The corresponding node, Indicates the first The first frame of the image Feature vectors of facial regions The corresponding node.

[0101] The region-dimensional hyperedge is constructed for different facial regions within the same frame node, and its expression is as follows:

[0102] ;

[0103] in, Indicates the first The first frame of the image Feature vectors of facial regions The corresponding node, Indicates the first The first frame of the image Feature vectors of facial regions The corresponding node.

[0104] The similarity-based hyperedge construction method selects the node most similar to the given node using a node similarity function. Given a set of nodes, similarity hyperedges are obtained; the similarity can be achieved through... Calculate; the expression for the similarity hyperedge is:

[0105] ;

[0106] in, Indicates the first 1 node Indicates and Most similar A set of nodes, express The Similar nodes, Indicate The Similar nodes, Indicate The Similar nodes; Represents a node eigenvectors, Represents a node eigenvectors, Represents norm operations.

[0107] In step S4, refer to Figure 5 As shown, the hyperbolic hypergraph learning module includes a hyperbolic exponential mapping layer, a hyperbolic learning module, and a hyperbolic logarithmic mapping layer;

[0108] Semantic classification features are extracted from the spatiotemporal hypergraph using a hyperbolic hypergraph learning module, including:

[0109] Using the hyperbolic exponential mapping layer, the feature vectors of the spatiotemporal hypergraph are mapped from Euclidean space to hyperbolic space to obtain the mapped features;

[0110] Based on the mapped features, semantic classification features are extracted using the hyperbolic learning module to obtain hyperbolic space classification features.

[0111] Using the hyperbolic logarithmic mapping layer, the hyperbolic space classification features are mapped back to Euclidean space to obtain the classification features.

[0112] The hyperbolic learning module includes L sets of cascaded hyperbolic feature transformation layers, hyperbolic neighborhood aggregation layers, and nonlinear activation layers. The hyperbolic feature transformation layers are used to linearly transform the features of hypergraph nodes. The hyperbolic neighborhood aggregation layers are used to aggregate information within the neighborhood of a node to obtain information about the surrounding environment. The nonlinear activation layers are used to introduce nonlinearity and dynamically adjust the curvature to improve the model's expressive and adaptive capabilities.

[0113] Semantic classification feature extraction using the hyperbolic learning module includes:

[0114] The hyperbolic feature transformation layer is used to perform a linear transformation on the mapped features to obtain linearly transformed features.

[0115] Based on the linear transformation features and the linear transformation features of the neighboring nodes of the node corresponding to the linear transformation features, the hyperbolic neighborhood aggregation layer is used for weighted aggregation to obtain the aggregated features;

[0116] The aggregated features are nonlinearly transformed using the nonlinear activation layer to obtain hyperbolic space classification features.

[0117] The expression for the hyperbolic exponential mapping layer is as follows:

[0118] ;

[0119] in, Indicates hyperbolic curvature. Represents the origin. Indicates is an exponential mapping function with as the origin and as the hyperbolic curvature,

[0120] The expression of the hyperbolic logarithm mapping layer is as follows:

[0121] ;

[0122] wherein, represents a hyperbolic space classification feature, represents a logarithm mapping function with as the origin and as the hyperbolic curvature.

[0123] The expression of the hyperbolic feature transformation layer is as follows:

[0124] ;

[0125] wherein, represents a feature vector of the th node in the th group of hyperbolic feature transformation layers, represents a feature vector of the th node in the th group of hyperbolic feature transformation layers, represents a weight matrix of the th group of hyperbolic feature transformation layers, represents a bias term of the th group of hyperbolic feature transformation layers, represents Möbius scalar multiplication.

[0126] The expression of the hyperbolic neighborhood aggregation layer is as follows:

[0127] ;

[0128] wherein, represents a feature of the th node, represents an aggregated feature of , and represents a hyperbolic curvature, represents a hyperedge between the th node and the th node, represents a weight of the hyperedge , and represents a feature set of neighborhood nodes of the th node, represents a logarithm mapping function with as the origin and as the hyperbolic curvature.a logarithmic mapping function with hyperbolic curvature, denotes an exponential mapping function with hyperbolic curvature, denotes an exponential mapping function with hyperbolic curvature, denotes an exponential mapping function with hyperbolic curvature.

[0129] The expression of the nonlinear activation layer is as follows:

[0130] ;

[0131] wherein, denotes the input of the nonlinear activation layer, denotes the output of the nonlinear activation layer, denotes the hyperbolic curvature of the first group of nonlinear activation layers, denotes the hyperbolic curvature of the first group of nonlinear activation layers, denotes an exponential mapping function with hyperbolic curvature, denotes an exponential mapping function with hyperbolic curvature, denotes an exponential mapping function with hyperbolic curvature, denotes a logarithmic mapping function with hyperbolic curvature, denotes a logarithmic mapping function with hyperbolic curvature, denotes a logarithmic mapping function with hyperbolic curvature, denotes an activation function.

[0132] In step S5, the classifier comprises one fully connected layer, the input size of the fully connected layer is the size of the classification feature, and the output size of the fully connected layer is 2, respectively representing the probability of being detected as true and false.

[0133] Embodiment two:

[0134] Based on the same inventive concept as embodiment one, the present embodiment discloses a face forgery detection system based on spatio-temporal hyperbolic hypergraph, comprising:

[0135] A data acquisition module is configured to acquire a face video to be detected for authenticity.

[0136] A face feature partition extraction module is configured to perform feature extraction on the face video to be detected for authenticity to obtain face local features and obtain a detection result.

[0137] A spatio-temporal node hypergraph construction module is configured to construct a spatio-temporal hypergraph according to the face local features.

[0138] A hyperbolic hypergraph learning module is configured to perform semantic classification feature extraction on the spatio-temporal hypergraph to obtain classification features.

[0139] A detection module is configured to input the classification features into a classifier to obtain a detection result.

[0140] The specific function implementation of each module above can refer to the related content in the method of embodiment one, and will not be described herein.

[0141] Those skilled in the art should understand that embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0142] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 a system for implementing the functions specified in one or more flows and / or blocks.

[0143] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction system that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 a system for implementing the functions specified in one or more flows and / or blocks.

[0144] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 a system for implementing the functions specified in one or more flows and / or blocks.

[0145] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative, but not restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, and these all belong to the protection of the present application.

Claims

1. A face forgery detection method based on spatio-temporal hypergraph hyperbolic, characterized in that, The method comprises the following steps: inputting a face video to be detected into a trained face forgery detection model: extracting features of the face video to be detected by a face feature partitioning module to obtain local face features; constructing a spatio-temporal hypergraph according to the local face features by a spatio-temporal hypergraph construction module; and extracting classification features from the spatio-temporal hypergraph by a hyperbolic hypergraph learning module; inputting the classification features into a classifier to obtain a detection result of true or false; the spatio-temporal hypergraph construction module uses the local face features as nodes, constructs hyperedges by a rule-based hyperedge construction method and a similarity-based hyperedge construction method, and obtains a spatio-temporal hypergraph; the expression of the spatio-temporal hypergraph is as follows: ， wherein, denotes a set of nodes, , denotes an i-th node, denotes a number of frames of a face video to be detected for authenticity, denotes a total number of face regions, denotes a set of hyper-edges, denotes a set of hyper-edges, , denotes a time-dimension hyper-edge, denotes a region-dimension hyper-edge, denotes a similarity hyper-edge; the rule-based hyperedge construction method establishes a connection between nodes in the time dimension and the region dimension to obtain time dimension hyperedges and region dimension hyperedges; the time dimension hyperedge is constructed for nodes of different frames with the same face region, and the expression of the time dimension hyperedge is as follows: ， wherein, represents the frame image, the feature vector of the corresponding node, represents the frame image, the feature vector of the corresponding node; the region dimension hyperedge is constructed for different face regions of nodes in the same frame, and the expression of the region dimension hyperedge is as follows: ， wherein, represents the feature vector of the face region in the corresponding node, represents the feature vector of the face region in the corresponding node; The similarity-based super-edge construction method selects, through a node similarity function, a node most similar to the node to obtain a similarity super-edge. ​ the expression of the similarity hyperedge is as follows: ， wherein, represents the i-th node, represents a set of the k most similar nodes to represents the i-th similar node of represents the i-th similar node of represents the i-th similar node of represents the i-th similar node of represents the i-th similar node of represents the i-th similar node of represents the i-th similar node of represents the i-th similar node of​​​​ 2. The face forgery detection method based on spatio-temporal hypergraph according to claim 1, characterized in that, the face feature partitioning module comprises a face region extraction module and a local feature extraction module; The face region extraction module includes a face key point detector and a face region segmenter; the face key point detector is based on the authenticity of the face video to be detected. Locate each frame of the image. Key points The facial region segmenter is based on key points. Classify faces into Facial area ;in, , , , This indicates the number of frames in the face video to be tested for authenticity. Indicates the frame number. Indicates the first Frame image, Indicates the first The first frame of the image One key point, Indicates the first The first frame of the image One facial area; The local feature extraction module extracts features based on each facial region. Using pre-trained convolutional neural networks Feature extraction is performed to obtain the facial region. eigenvectors Output the local facial features ,in, , , Indicates the first In the frame image A set of feature vectors for each facial region .

3. The face forgery detection method based on spatio-temporal hypergraph according to claim 1, characterized in that, the hyperbolic hypergraph learning module comprises a hyperbolic exponential mapping layer, a hyperbolic learning module, and a hyperbolic logarithmic mapping layer; the hyperbolic hypergraph learning module extracts semantic classification features from the spatio-temporal hypergraph, which comprises the following steps: the hyperbolic exponential mapping layer is used to map the feature vector of the spatio-temporal hypergraph from Euclidean space to hyperbolic space to obtain a mapped feature; the hyperbolic learning module is used to extract semantic classification features according to the mapped feature to obtain hyperbolic space classification features; the hyperbolic logarithmic mapping layer is used to map the hyperbolic space classification features back to Euclidean space to obtain classification features.

4. The face forgery detection method based on spatio-temporal hypergraph according to claim 3, characterized in that, The expression of the hyperbolic exponential mapping layer is as follows: ， wherein, represents a hyperbolic curvature, represents the origin, represents an exponential mapping function with as the origin and as the hyperbolic curvature, represents a characteristic vector of a spacetime hypergraph in Euclidean space; The expression of the hyperbolic logarithmic mapping layer is as follows: ， wherein, represents a hyperbolic space classification feature, represents a logarithmic mapping function with as the origin and as the hyperbolic curvature.

5. The face forgery detection method based on spatio-temporal hypergraph according to claim 3, characterized in that, The hyperbolic learning module at least comprises a group of hyperbolic feature transformation layers, hyperbolic neighborhood aggregation layers, and nonlinear activation layers connected in series; the hyperbolic learning module is used to extract semantic classification features, which comprises the following steps: the hyperbolic feature transformation layer is used to perform linear transformation on the mapped feature to obtain a linearly transformed feature; the hyperbolic neighborhood aggregation layer is used to perform weighted aggregation on the linearly transformed feature and the linearly transformed features of the neighborhood nodes of the corresponding node to obtain an aggregated feature; the nonlinear activation layer is used to perform nonlinear transformation on the aggregated feature to obtain hyperbolic space classification features.

6. The face forgery detection method based on spatio-temporal hypergraph according to claim 5, characterized in that, The expression of the hyperbolic feature transformation layer is as follows: ， wherein, represents the eigenvector of the th node in the th hyperbolic characteristic transformation layer, represents the eigenvector of the th node in the th hyperbolic characteristic transformation layer, represents the weight matrix of the th hyperbolic characteristic transformation layer, represents the bias term of the th hyperbolic characteristic transformation layer, represents Möbius scalar multiplication; The expression of the hyperbolic neighborhood aggregation layer is as follows: ， wherein, represents a feature of the th node, represents a feature of the th node after aggregation, represents a hyperbolic curvature, represents a hyperedge between the th node and the th node, represents a weight of the hyperedge , represents a feature set of the neighborhood nodes of the th node, represents a logarithmic mapping function with the th node as the origin and the th node as the hyperbolic curvature, represents an exponential mapping function with the th node as the origin and the th node as the hyperbolic curvature; The expression of the nonlinear activation layer is as follows: ， wherein, denotes the input of a nonlinear activation layer, denotes the output of a nonlinear activation layer, denotes the hyperbolic curvature of the group of nonlinear activation layers, denotes the hyperbolic curvature of the group of nonlinear activation layers, denotes an exponential mapping function with as origin and as hyperbolic curvature, denotes a logarithmic mapping function with as origin and as hyperbolic curvature, denotes an activation function.

7. The method for face forgery detection based on spatio-temporal hypergraph according to claim 1, characterized in that, The classifier comprises one fully connected layer, the input size of the fully connected layer is the size of the classification features, and the output size of the fully connected layer is 2, representing the probabilities of true and false detection, respectively; the training process of the face forgery detection model comprises the following steps: Construct a training data set, the training data set includes real face video data and face video data generated by using a fake technology, all video data are equipped with corresponding labels, used to distinguish real data and fake data; According to the training data set, the cross entropy loss of the face fake detection model is calculated; Based on the gradient descent method, the face fake detection model is iteratively updated and trained to minimize the cross entropy loss, and a trained face fake detection model is obtained; The formula of the cross entropy loss is: ， wherein, denotes a label, denotes a detection result, denotes a detection result and a label between the loss.

8. A face forgery detection system based on spatio-temporal hypergraph, characterized in that, The face fake detection method based on the spatio-temporal hyperbolic hypergraph according to any one of claims 1-7 comprises: A data acquisition module is configured to acquire a face video to be detected for authenticity; A face feature partition extraction module is configured to extract features from the face video to be detected for authenticity, obtain face local features, and obtain a detection result; A spatio-temporal node hypergraph construction module is configured to construct a spatio-temporal hypergraph according to the face local features; A hyperbolic hypergraph learning module is configured to extract classification features from the spatio-temporal hypergraph; A detection module is configured to input the classification features into a classifier to obtain a detection result.

Images