Face video recognition method and system
By extracting heart rate and texture features from the video frame image sequence of face videos, and combining them with a lightweight deep convolutional neural network and classifier, the problem of low accuracy in face video authenticity detection in existing technologies is solved, achieving more efficient authenticity discrimination and identification of fake face videos.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INSTITUTE OF INFORMATION ENGINEERING CHINESE ACADEMY OF SCIENCES
- Filing Date
- 2023-05-12
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies only consider the differences between face video frames and do not perform a global analysis of the entire video, resulting in a low accuracy rate in detecting the authenticity of face videos.
By acquiring video frame image sequences of facial videos, human heart rate and texture features are extracted, and a lightweight deep convolutional neural network model and motion estimation method are used, combined with a classifier and a fusion discrimination model to distinguish between real and fake faces.
It improves the accuracy of distinguishing between real and fake face videos, reduces computational complexity, and enhances the ability to identify fake face videos, especially the ability to defend against 3D face mask attacks.
Smart Images
Figure CN116863545B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of information security technology, and in particular to a face video recognition method and system. Background Technology
[0002] With the widespread adoption of facial recognition technology, it provides a convenient and quick way to verify identities in scenarios such as mobile payments, identification at important locations, and authorization of mobile devices, simplifying complex authentication processes. However, facial recognition also faces numerous risks. For example, it can be used to synthesize faces using video synthesis technology or to deceive users with realistic 3D face masks, easily bypassing facial recognition system verification and leading to the leakage of important information or financial losses. To address the problem of fake facial videos, it is necessary to verify the authenticity of the input video before facial recognition.
[0003] Common methods for forging videos include attackers wearing 3D face masks based on the victim's real face, and using video synthesis technology to create fake videos. To prevent fraudulent videos using forged faces, researchers have proposed various methods for video analysis and authentication. Currently common methods include texture-based methods, motion-based methods, remote photoplethysmography (RP-PPR) methods, and deep learning-based methods.
[0004] Existing methods for detecting forged video images based on inter-frame quantization parameter intensity values and facial motion unit (FMU)-based methods directly analyze and detect images using data extracted by the FMU acquisition program. These methods improve upon the previous approach of mapping data to a high-dimensional space via kernel functions, instead employing a new approach of data dimensionality reduction. This avoids the need to extract additional, undetectable data from the facial coding system. Furthermore, based on the detection results, they compare the images with a face database to identify the original individuals in the forged videos. However, these methods only consider information changes between video frames and do not take into account the overall global information of the video, which can easily lead to the loss of some key information. Summary of the Invention
[0005] The face video recognition method and system provided by this invention are used to solve the problem that the existing technology only considers the differences between face video frames and does not perform global analysis of the entire video, resulting in the loss of key information and thus the low accuracy of face video authenticity detection.
[0006] The present invention provides a face video recognition method, comprising:
[0007] Obtain the video frame image sequence corresponding to the face video, wherein the video frame image sequence is composed of multiple video frame images, and the video frame images are obtained by processing the face video frame by frame;
[0008] Based on the video frame image sequence, obtain the human heart rate features and texture features in the face video;
[0009] The authenticity of the face video is determined based on the human heart rate characteristics and the texture characteristics.
[0010] According to a face video recognition method provided by the present invention, the step of obtaining human heart rate features and texture features in the face video based on the video frame image sequence includes:
[0011] The video frame image sequence is input into a deep convolutional neural network model, and the video frame image sequence is processed based on the differential normalization layer in the deep convolutional neural network model to obtain multiple first video frame images.
[0012] Multiple first video frame images are input into multiple depthwise separable convolutional layers in the deep convolutional neural network model, and the human heart rate features are obtained based on the output of the target depthwise separable convolutional layer. The deep convolutional neural network model is obtained by training a lightweight convolutional neural network model using the RPPG heart rate measurement dataset. The target depthwise separable convolutional layer is a depthwise separable convolutional layer connected to the fully connected layer in the deep convolutional neural network model.
[0013] Extract the second video frame image corresponding to the key video frame in the video frame image sequence;
[0014] The second video frame image is input into a target convolutional neural network model to obtain the texture features. The target convolutional neural network model is used to extract the texture features of the second video frame image.
[0015] According to a face video recognition method provided by the present invention, determining the authenticity of the face video based on the human heart rate features and the texture features includes:
[0016] The human heart rate feature is input into a classifier to obtain a first discrimination score. The classifier is used to output the first discrimination score, which is used to characterize the authenticity of the face video based on the human heart rate feature.
[0017] The second video frame image is input into the target convolutional neural network model to obtain the second discrimination score. The target convolutional neural network model is also used to output the second discrimination score, which is used to characterize the authenticity of the face video based on the texture features.
[0018] The authenticity of the face video is determined based on the first discrimination score and the second discrimination score.
[0019] According to a face video recognition method provided by the present invention, determining the authenticity of the face video based on the first discrimination score and the second discrimination score includes:
[0020] The final discriminant score is calculated based on the first discriminant score, the weight of the first discriminant score, the second discriminant score, and the weight of the second discriminant score.
[0021] The authenticity of the facial video is determined based on the final discrimination score.
[0022] According to a face video recognition method provided by the present invention, the step of extracting a second video frame image corresponding to a key video frame in the video frame image sequence includes:
[0023] The motion vectors between adjacent video frames in the video frame image sequence are calculated using a motion estimation method.
[0024] The preset number of video frames with the largest motion vector are used as the second video frame image corresponding to the key video frame.
[0025] According to a face video recognition method provided by the present invention, determining the authenticity of the face video based on the final discrimination score includes:
[0026] If the final discrimination score is greater than a preset value, then the face video is determined to be genuine;
[0027] Otherwise, the facial video is determined to be fake.
[0028] The present invention also provides a face video recognition system, comprising: a data acquisition module, a feature extraction module, and a authenticity discrimination module;
[0029] The data acquisition module is used to acquire a video frame image sequence corresponding to the face video. The video frame image sequence is composed of multiple video frame images, which are obtained by processing the face video frame by frame.
[0030] The feature extraction module is used to obtain human heart rate features and texture features in the face video based on the video frame image sequence;
[0031] The authenticity determination module is used to determine the authenticity of the face video based on the human heart rate characteristics and the texture characteristics.
[0032] The present invention also provides an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement the face video recognition method as described above.
[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the face video recognition method as described above.
[0034] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the face video recognition method as described above.
[0035] The face video recognition method and system provided by this invention obtains the video frame image sequence corresponding to the face video, and obtains the human heart rate feature and texture feature in the face video based on the video frame image sequence, and combines the human heart rate feature and texture feature to judge the authenticity of the face video, thereby improving the accuracy of face video authenticity judgment. Attached Figure Description
[0036] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0037] Figure 1 This is one of the flowcharts illustrating the face video recognition method provided by the present invention;
[0038] Figure 2 This is the second flowchart illustrating the face video recognition method provided by the present invention;
[0039] Figure 3 This is a schematic diagram of the structure of the face video recognition system provided by the present invention;
[0040] Figure 4 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation
[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0042] Figure 1 This is one of the flowcharts illustrating the face video recognition method provided by the present invention, such as... Figure 1 As shown, the method includes:
[0043] Step 110: Obtain the video frame image sequence corresponding to the face video. The video frame image sequence is composed of multiple video frame images, which are obtained by processing the face video frame by frame.
[0044] Step 120: Based on the video frame image sequence, obtain the human heart rate features and texture features in the face video;
[0045] Step 130: Determine the authenticity of the face video based on the human heart rate characteristics and the texture characteristics.
[0046] It should be noted that the above method can be implemented by computer equipment.
[0047] Optionally, the sequence of video frame images corresponding to the face video is obtained by preprocessing the face video as follows:
[0048] The original face video is segmented into frames, converting the complete face video into individual images. Each image is then cropped to a uniform size, resulting in corresponding video frame images. The sequence of multiple video frame images is the video frame image sequence.
[0049] The heart rate features in this facial video can be extracted from the corresponding video frame image sequence using a non-contact heart rate measurement method based on remote photoplethysmography (RPPG). When light shines on the human body surface, it is projected or reflected. Because the human heartbeat causes periodic changes in blood vessel volume, the heart rate features can be calculated by extracting the signals caused by these changes in the video.
[0050] The texture features in the human body video can be specifically defined as the texture features of the video frame images corresponding to the key video frames in the video frame image sequence.
[0051] Optionally, the human heart rate and texture features of the obtained face video can be used to identify the face video, thereby determining whether the face video is real or fake.
[0052] The face video recognition method provided by this invention obtains the video frame image sequence corresponding to the face video, and obtains the human heart rate feature and texture feature in the face video based on the video frame image sequence. The authenticity of the face video is judged by combining the human heart rate feature and texture feature, thereby improving the accuracy of face video authenticity judgment.
[0053] Furthermore, in one embodiment, obtaining the human heart rate features and texture features in the face video based on the video frame image sequence may specifically include:
[0054] The video frame image sequence is input into a deep convolutional neural network model, and the video frame image sequence is processed based on the differential normalization layer in the deep convolutional neural network model to obtain multiple first video frame images.
[0055] Multiple first video frame images are input into multiple depthwise separable convolutional layers in the deep convolutional neural network model, and the human heart rate features are obtained based on the output of the target depthwise separable convolutional layer. The deep convolutional neural network model is obtained by training a lightweight convolutional neural network model using the RPPG heart rate measurement dataset. The target depthwise separable convolutional layer is a depthwise separable convolutional layer connected to the fully connected layer in the deep convolutional neural network model.
[0056] Extract the second video frame image corresponding to the key video frame in the video frame image sequence;
[0057] The second video frame image is input into a target convolutional neural network model to obtain the texture features. The target convolutional neural network model is used to extract the texture features of the second video frame image.
[0058] Optionally, existing deep face forgery video detection methods based on rPPG signals include: acquiring face sequences from the video; selecting specific regions of interest (ROIs) on the face and extracting rPPG signals using a green single-channel method; and using a CNN-based classifier to identify forgeries based on the acquired rPPG signals. This method leverages the difficulty of simulating human heart rate signals in most current forgery methods by extracting rPPG signals from specific parts of the face in the video for forgery detection. It can effectively detect face forgery videos generated by various forgery methods. However, this method requires additional face extraction operations, increasing the overall complexity, and the network model has a large number of parameters, placing high demands on computing power.
[0059] The existing technology provides a method and system for detecting deepfake faces in videos based on image feature fusion. This method includes the following steps: first, segmenting the video into frames; extracting the face region from the video to obtain a face image; preprocessing the spatial domain data, frequency domain data, and local gravity pattern (PLGF) map of the face image separately, adjusting them to achieve a unified size requirement, to obtain the face image's spatial domain features, frequency domain features, and PLGF features; concatenating and fusing these features using a network structure, inputting them into a two-layer Long Short-Term Memory (LSTM) network structure to extract the final fused features, which are then input into a fully connected layer for classification; finally, the output is judged after passing through a Softmax activation function. This method uses different information domain data preprocessing methods before feature fusion, ensuring good detection results within the database while significantly reducing the average error rate of cross-database detection. However, this method requires considering multiple dimensions of video information, calculating their features separately, and then fusing the features, making the overall calculation process quite cumbersome.
[0060] Existing face spoofing video detection methods based on dual-stream neural network models preprocess the collected videos; perform face detection, tracking, and recognition on the preprocessed videos, store the detected faces to ensure temporal continuity of the same face; and feed the temporally continuous faces into a 3D convolutional search network for real / fake face detection and output the results, achieving fast and accurate detection of spoofed faces. However, this method uses a three-dimensional convolutional neural network model for spoofing face recognition, which is complex to train and optimize, has large model parameters, slow training speed, and high equipment requirements.
[0061] Based on this, the present invention distinguishes between genuine and fake input face videos by fusing the human heart rate features of the face video with the texture features of the face video itself. Therefore, the present invention consists of two branch models (heart rate feature model and texture feature model) and a fusion discrimination model.
[0062] The heart rate feature model is used to extract human heart rate features from face videos; the texture feature model is used to extract texture features from face videos; and the fusion discrimination model is used to fuse the human heart rate features and the texture features to determine the authenticity of the face video.
[0063] Alternatively, a deep convolutional neural network model for extracting heart rate information can be trained to extract human heart rate features from face videos.
[0064] A lightweight two-dimensional convolutional neural network (CNN) model is used to extract heart rate information. To ensure the accuracy of heart rate extraction and reduce the error of the lightweight CNN model, pre-training is required. Specifically, the architecture of the lightweight CNN model is constructed and pre-trained on the publicly available RPPG heart rate measurement dataset, while retaining the weight parameters of the lightweight CNN model. Addressing the issues of complex training and optimization of three-dimensional convolutions, and the large number of parameters and computational load, this invention proposes a recognition method based on a lightweight CNN model. This method utilizes lightweight depthwise separable convolutions to replace conventional CNNs, reducing not only the model's parameters and computational load but also avoiding any loss of model accuracy.
[0065] We use a trained lightweight convolutional neural network model to extract human heart rate features from face videos.
[0066] Heart rate information is one-dimensional and not suitable as direct input for identifying fake face videos. Therefore, it needs to be modeled to extract heart rate features. Here, a trained deep convolutional neural network model is used to extract heart rate features, and the output of the target depth separable convolutional layer of the deep convolutional neural network model is used as the human heart rate feature of the entire face video.
[0067] Optionally, multiple video frame images from the video frame image sequence corresponding to the face video are input into a trained lightweight convolutional neural network model (i.e., a deep convolutional neural network model) for processing. The differential normalization layer in the deep convolutional neural network model performs differential operations on adjacent video frame images to obtain multiple video frame images after differential operations, i.e., the first video frame image.
[0068] The multiple first video frame images are input into multiple depthwise separable convolutional layers (e.g., 6 layers) in a deep convolutional neural network model. The output of the target depthwise separable convolutional layer (blood volume pulse (BVP) signal) is used to obtain the human heart rate features of the face video based on the BVP signal. The target depthwise separable convolutional layer can be specifically a depthwise separable convolutional layer connected to a fully connected layer in the deep convolutional neural network model. The depthwise separable convolutional layer adopts a structure similar to MobileNet v2, and the attention layer is embedded in the depthwise separable convolutional layer, which is stacked as the basic block of the lightweight convolutional neural network model.
[0069] The attention layer used here can be specifically a lightweight channel attention module, EfficientChannel Attention (ECA). The ECA module introduces local cross-channel interactions, reducing model complexity and using one-dimensional convolutions to extract features without needing to reduce the number of channels. This effectively captures cross-channel interaction information, thus achieving high-efficiency channels in deep convolutional neural networks.
[0070] Optionally, by extracting the video frame images corresponding to key video frames in the video frame image sequence, a second video frame image is obtained, and this second video frame image is input into the target convolutional neural network model to obtain the texture features of the face video. Here, instead of using every single video frame image in the video frame image sequence corresponding to the face video as input to the target convolutional neural network model, the second video frame image (the video frame image corresponding to the key frame of the input face video) is extracted and used as input to the target convolutional neural network model. This reduces the computational cost of the model without compromising its accuracy.
[0071] Specifically, this target convolutional neural network model can be used to extract and classify texture features in the second video frame image. Its training process includes:
[0072] The dataset used is a publicly available face deception dataset, which contains various face spoofing attack videos and AI-synthesized face videos.
[0073] The video frame images corresponding to the keyframes of face videos in the publicly available face anti-spoofing dataset are extracted and divided into training, testing, and validation sets in a 7:2:1 ratio. The convolutional neural network model is trained based on the training, testing, and validation sets to obtain the target convolutional neural network model. After training, the target convolutional neural network model can extract and classify the texture features of face videos, and finally obtain a score based on the texture features of face videos, namely the second discriminant score.
[0074] Furthermore, in one embodiment, extracting the second video frame image corresponding to the key video frame in the video frame image sequence may specifically include:
[0075] The motion vectors between adjacent video frames in the video frame image sequence are calculated using a motion estimation method.
[0076] The preset number of video frames with the largest motion vector are used as the second video frame image corresponding to the key video frame.
[0077] Optionally, the present invention uses motion estimation to extract key frames in face videos. By calculating the motion vectors between adjacent video frame images, a preset number of frames with the largest motion vectors are selected as key frames, and the video frame images corresponding to these key frames are input as second video frame images into the target convolutional neural network model.
[0078] This invention provides a face video recognition method that addresses the problem of complex operation processes requiring additional face extraction. It proposes a lightweight convolutional neural network model with an added attention mechanism module. With this module, the network assigns different weights to the input content, focusing on calculating the face portion and ignoring irrelevant parts, thus improving the accuracy of face video authenticity detection. To address the cumbersome process of considering multiple dimensions of video information, calculating features separately, and then fusing them, this invention proposes a face video authenticity detection method that integrates human heart rate and texture features. This only requires considering the influence of two features, facilitating model fusion and reducing computational complexity. Furthermore, using human heart rate features to assist in identifying genuine face videos enhances the model's robustness and demonstrates excellent performance against specific types of attacks, such as 3D face mask attacks.
[0079] Furthermore, in one embodiment, determining the authenticity of the facial video based on the human heart rate characteristics and the texture characteristics may specifically include:
[0080] The human heart rate feature is input into a classifier to obtain a first discrimination score. The classifier is used to output the first discrimination score, which is used to characterize the authenticity of the face video based on the human heart rate feature.
[0081] The second video frame image is input into the target convolutional neural network model to obtain the second discrimination score. The target convolutional neural network model is also used to output the second discrimination score, which is used to characterize the authenticity of the face video based on the texture features.
[0082] The authenticity of the face video is determined based on the first discrimination score and the second discrimination score.
[0083] Optionally, the classifier can be specifically used to classify the authenticity of a face video based on the human heart rate features in the input face video. Specifically, the obtained human heart rate features are input into the classifier, and the classifier determines whether the face video is a real face video or a fake face video based on the discriminant score (i.e., the first discriminant score) output by the classifier. The distinction can be made based on the value of the first discriminant score. For example, if the value of the first discriminant score is within the first interval, the face video is considered to be a fake face video; if the value of the first discriminant score is within the second interval, the face video is considered to be a real face video. The first interval and the second interval can be flexibly set according to the actual situation. For example, the first interval is (0, 0, 5] and the second interval is (0, 5, 1).
[0084] It should be noted that this classifier can be specifically a binary classifier, and it can specifically use the random forest algorithm as the binary classifier.
[0085] Optionally, the target convolutional neural network model can also be specifically used to obtain the texture features of the face video based on the second video frame image corresponding to the keyframe in the input face video and classify them. The classification result of the texture features can be specifically used to classify the authenticity of the face video. Specifically, the obtained second video frame image is input into the target convolutional neural network model, and the face video is judged to be a real face video or a fake face video based on the discriminant score (i.e., the second discriminant score) output by the target convolutional neural network model. It can be specifically distinguished based on the value of the second discriminant score. For example, if the value of the second discriminant score is within the third interval, the face video is considered to be a fake face video. If the value of the second discriminant score is within the fourth interval, the face video is considered to be a real face video. The third and fourth intervals can be flexibly set according to the actual situation. For example, the third interval is (0, 0.5] and the fourth interval is (0.5, 1).
[0086] The first and second discriminant scores are fused using a fusion discriminant model to determine whether the face video is a real face video or a fake face video.
[0087] Furthermore, in one embodiment, determining the authenticity of the face video based on the first discrimination score and the second discrimination score may specifically include:
[0088] The final discriminant score is calculated based on the first discriminant score, the weight of the first discriminant score, the second discriminant score, and the weight of the second discriminant score.
[0089] The authenticity of the facial video is determined based on the final discrimination score.
[0090] Optionally, based on the heart rate feature model and the texture feature model, a first discrimination score for judging the authenticity of the face video based on human heart rate features and a second discrimination score for judging the authenticity of the face video based on texture features can be obtained respectively. The heart rate feature model and the texture feature model are then fused using this fusion discrimination model to obtain the final discrimination score. The fusion formula used in this invention is as follows:
[0091] S=ω*S RPPG +(1-ω)*S CNN ;
[0092] Where S represents the final discriminant score, ω represents the weight of the first discriminant score, and S RPPG S represents the first discriminant score. CNN (1-ω) represents the second discriminant, and (1-ω) represents the weight of the second discriminant. The value of ω can be flexibly set. In this invention, ω is 0.3.
[0093] Based on the final discrimination score S, it is determined whether the face video is a real face video or a fake face video.
[0094] Furthermore, in one embodiment, determining the authenticity of the facial video based on the final discrimination score may specifically include:
[0095] If the final discrimination score is greater than a preset value, then the face video is determined to be genuine;
[0096] Otherwise, the facial video is determined to be fake.
[0097] Optionally, the final discrimination score S can be used for discrimination. If S is greater than a preset value (e.g., 0.5), it is considered to be a real face video. If S is less than or equal to 0.5, it is considered to be a fake face video, that is, the face video is fake.
[0098] Figure 2 This is the second flowchart illustrating the face video recognition method provided by the present invention, as shown below. Figure 2 As shown, the method includes:
[0099] The input face video is preprocessed to obtain a sequence of video frame images;
[0100] The video frame image sequence is input into the heart rate feature model and the texture feature model, respectively;
[0101] The first video frame image is obtained by performing differential processing on the video frame image sequence based on the trained deep convolutional neural network model.
[0102] The first video frame image is input into a trained deep convolutional neural network model to extract the human heart rate features of the face video;
[0103] The heart rate feature of this human body is input into the classifier to obtain the first discriminant score;
[0104] Based on this texture feature model, the second video frame image corresponding to the key video frame is extracted;
[0105] The second video frame image is input into the target convolutional neural network model to obtain the texture features of the face video and classify the texture features to obtain the second discriminant score;
[0106] The first and second discriminant scores are fused based on the fusion discriminant model to obtain the final discriminant score;
[0107] If the final discrimination score is greater than the preset value of 0.5, then the face video is determined to be a real face video;
[0108] Otherwise, the face video is determined to be a fake face video.
[0109] The face video recognition method provided by this invention can detect whether the original face video or the face video captured by the camera is real or fake, and provide a prompt. It can serve as the first barrier for face recognition, preventing face deception attacks and fake face attacks.
[0110] The face video recognition system provided by the present invention will be described below. The face video recognition system described below can be referred to in correspondence with the face video recognition method described above.
[0111] Figure 3 This is a schematic diagram of the facial video recognition system provided by the present invention, as shown below. Figure 3 As shown, it includes:
[0112] Data acquisition module 310, feature extraction module 311, and authenticity discrimination module 312;
[0113] The data acquisition module 310 is used to acquire a video frame image sequence corresponding to the face video. The video frame image sequence is composed of multiple video frame images, which are obtained by processing the face video frame by frame.
[0114] The feature extraction module 311 is used to obtain human heart rate features and texture features in the face video based on the video frame image sequence;
[0115] The authenticity determination module 312 is used to determine the authenticity of the face video based on the human heart rate characteristics and the texture characteristics.
[0116] The face video recognition system provided by this invention obtains the video frame image sequence corresponding to the face video, and based on the video frame image sequence, obtains the human heart rate features and texture features in the face video, and combines the human heart rate features and texture features to judge the authenticity of the face video, thereby improving the accuracy of face video authenticity judgment.
[0117] Figure 4 This is a schematic diagram of the physical structure of an electronic device provided by the present invention, such as... Figure 4 As shown, the electronic device may include a processor 410, a communication interface 411, a memory 412, and a bus 413. The processor 410, communication interface 411, and memory 412 communicate with each other via the bus 413. The processor 410 can call logical instructions from the memory 412 to execute the following methods:
[0118] Obtain the video frame image sequence corresponding to the face video, wherein the video frame image sequence is composed of multiple video frame images, and the video frame images are obtained by processing the face video frame by frame;
[0119] Based on the video frame image sequence, obtain the human heart rate features and texture features in the face video;
[0120] The authenticity of the face video is determined based on the human heart rate characteristics and the texture characteristics.
[0121] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer power supply (which may be a personal computer, server, or network power supply, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0122] Furthermore, this invention discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when these instructions are executed by a computer, the computer can perform the face video recognition method provided in the above-described method embodiments, for example including:
[0123] Obtain the video frame image sequence corresponding to the face video, wherein the video frame image sequence is composed of multiple video frame images, and the video frame images are obtained by processing the face video frame by frame;
[0124] Based on the video frame image sequence, obtain the human heart rate features and texture features in the face video;
[0125] The authenticity of the face video is determined based on the human heart rate characteristics and the texture characteristics.
[0126] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the face video recognition methods provided in the above embodiments, including, for example:
[0127] Obtain the video frame image sequence corresponding to the face video, wherein the video frame image sequence is composed of multiple video frame images, and the video frame images are obtained by processing the face video frame by frame;
[0128] Based on the video frame image sequence, obtain the human heart rate features and texture features in the face video;
[0129] The authenticity of the face video is determined based on the human heart rate characteristics and the texture characteristics.
[0130] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer power supply (which may be a personal computer, server, or network power supply, etc.) to execute the methods described in various embodiments or some parts of the embodiments.
[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A face video recognition method, characterized in that, include: Obtain the video frame image sequence corresponding to the face video, wherein the video frame image sequence is composed of multiple video frame images, and the video frame images are obtained by processing the face video frame by frame; Based on the video frame image sequence, obtain the human heart rate features and texture features in the face video; The authenticity of the face video is determined based on the human heart rate characteristics and the texture characteristics. The step of obtaining human heart rate features and texture features in the face video based on the video frame image sequence includes: The video frame image sequence is input into a deep convolutional neural network model, and the video frame image sequence is processed based on the differential normalization layer in the deep convolutional neural network model to obtain multiple first video frame images. Multiple first video frame images are input into multiple depthwise separable convolutional layers in the deep convolutional neural network model, and the human heart rate features are obtained based on the output of the target depthwise separable convolutional layer. The deep convolutional neural network model is obtained by training a lightweight convolutional neural network model using the RPPG heart rate measurement dataset. The target depthwise separable convolutional layer is a depthwise separable convolutional layer connected to the fully connected layer in the deep convolutional neural network model. Extract the second video frame image corresponding to the key video frame in the video frame image sequence; The second video frame image is input into the target convolutional neural network model to obtain the texture features. The target convolutional neural network model is used to extract the texture features of the second video frame image. Extracting the second video frame image corresponding to the key video frame in the video frame image sequence includes: The motion estimation method is used to calculate the motion vectors between adjacent video frames in the video frame image sequence; The preset number of video frames with the largest motion vector are used as the second video frame image corresponding to the key video frame.
2. The face video recognition method according to claim 1, characterized in that, Determining the authenticity of the face video based on the human heart rate characteristics and the texture characteristics includes: The human heart rate feature is input into a classifier to obtain a first discrimination score. The classifier is used to output the first discrimination score, which is used to characterize the authenticity of the face video based on the human heart rate feature. The second video frame image is input into the target convolutional neural network model to obtain the second discrimination score. The target convolutional neural network model is also used to output the second discrimination score, which is used to characterize the authenticity of the face video based on the texture features. The authenticity of the face video is determined based on the first discrimination score and the second discrimination score.
3. The face video recognition method according to claim 2, characterized in that, Determining the authenticity of the face video based on the first discrimination score and the second discrimination score includes: The final discriminant score is calculated based on the first discriminant score, the weight of the first discriminant score, the second discriminant score, and the weight of the second discriminant score. The authenticity of the facial video is determined based on the final discrimination score.
4. The face video recognition method according to claim 3, characterized in that, Determining the authenticity of the face video based on the final discrimination score includes: If the final discrimination score is greater than a preset value, then the face video is determined to be genuine; Otherwise, the facial video is determined to be fake.
5. A facial video recognition system, characterized in that, include: Data acquisition module, feature extraction module, and authenticity determination module; The data acquisition module is used to acquire a video frame image sequence corresponding to the face video. The video frame image sequence is composed of multiple video frame images, which are obtained by processing the face video frame by frame. The feature extraction module is used to obtain human heart rate features and texture features in the face video based on the video frame image sequence; The authenticity determination module is used to determine the authenticity of the face video based on the human heart rate characteristics and the texture characteristics. The step of obtaining human heart rate features and texture features in the face video based on the video frame image sequence includes: The video frame image sequence is input into a deep convolutional neural network model, and the video frame image sequence is processed based on the differential normalization layer in the deep convolutional neural network model to obtain multiple first video frame images. Multiple first video frame images are input into multiple depthwise separable convolutional layers in the deep convolutional neural network model, and the human heart rate features are obtained based on the output of the target depthwise separable convolutional layer. The deep convolutional neural network model is obtained by training a lightweight convolutional neural network model using the RPPG heart rate measurement dataset. The target depthwise separable convolutional layer is a depthwise separable convolutional layer connected to the fully connected layer in the deep convolutional neural network model. Extract the second video frame image corresponding to the key video frame in the video frame image sequence; The second video frame image is input into the target convolutional neural network model to obtain the texture features. The target convolutional neural network model is used to extract the texture features of the second video frame image. Extracting the second video frame image corresponding to the key video frame in the video frame image sequence includes: The motion estimation method is used to calculate the motion vectors between adjacent video frames in the video frame image sequence; The preset number of video frames with the largest motion vector are used as the second video frame image corresponding to the key video frame.
6. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the face video recognition method according to any one of claims 1 to 4.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the face video recognition method as described in any one of claims 1 to 4.
8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the face video recognition method as described in any one of claims 1 to 4.