Face recognition method and device based on short video

Abstract

The application provides a short video-based face recognition method and device. The method comprises: acquiring a short video containing a target face and performing face detection to obtain a face video image; enhancing a face change region image based on band-pass filtering of the face video image; extracting face change features for the enhanced image, and identifying whether the face is a real face or a false face based on the extracted face change features; if the face is a real face, performing feature extraction on the face video image and matching the face features with face features saved in a database to identify the identity. The application can effectively extract tiny face change features by enhancing the face change region image, and can effectively detect face recognition attacks and improve the accuracy and reliability of face recognition by detecting real and false faces based on the extracted face change features.

Description

Technical Field

[0001] This invention belongs to the field of facial recognition technology, specifically relating to a facial recognition method and device based on short videos. Background Technology

[0002] Identity verification is not only a common challenge in daily life, but also crucial in national defense, scientific research, security, and intelligent manufacturing. Facial recognition technology, a major branch of identity verification, is widely used due to its high accuracy and ease of use. However, the widespread adoption of facial recognition systems also means that breaching them could generate substantial illegal profits. This has led criminals to employ various methods to attack facial recognition systems, posing significant security risks to users' information and property.

[0003] Currently, existing attack methods are mainly divided into three categories: photo attacks, 3D model attacks, and video replay attacks. Photo attacks are the most common type. With the development and popularization of the internet, photos containing faces are frequently published online by users. Criminals can easily obtain users' facial images, print them out, or play them on electronic screens to attack facial recognition systems. 3D model attacks involve modeling the user's face to create a facial model or mask that resembles a real face, which can then be used to attack facial recognition systems. Current technology cannot perfectly reproduce a real face, and creating attack samples is difficult and not easy to implement. However, once a high-quality mask attack occurs, it can significantly impact the security of facial recognition systems. Video replay attacks refer to criminals obtaining user facial videos through social media platforms or by secretly filming, and then playing them on electronic screens to attack facial recognition systems. Because video replay attacks utilize some of the user's action information, they pose a higher threat to the system, especially effective against facial recognition systems based on short videos. Summary of the Invention

[0004] To address the aforementioned problems in the existing technology, the present invention provides a face recognition method and apparatus based on short videos.

[0005] To achieve the above objectives, the present invention adopts the following technical solution.

[0006] In a first aspect, the present invention provides a face recognition method based on short videos, comprising the following steps:

[0007] Acquire a short video containing the target face and perform face detection to obtain a face video image;

[0008] Enhancement of facial change regions is achieved by applying bandpass filtering to the facial video images;

[0009] Facial change features are extracted from the enhanced image, and the extracted facial change features are used to identify whether the face is real or fake.

[0010] If it is a real human face, the identity is identified by extracting features from the face video image and matching them with the facial features stored in the database.

[0011] Furthermore, the method for enhancing the image of facial change regions includes:

[0012] Each frame of the image is decomposed using the Laplacian pyramid to obtain images at different scales;

[0013] Bandpass filtering is performed on the image at each scale, and the frequency signal within the filter's passband is output.

[0014] The pixel values ​​of the filtered image are multiplied by a magnification factor, which is different for each scale.

[0015] After the magnified image pyramid is restored to a single image, it is added to the image before the decomposition, and the added images are combined to form a video.

[0016] Furthermore, the method for extracting facial change features from the enhanced image includes:

[0017] Convert the RGB space image to the YCbCr or HSV space image, and extract the luminance component from the Y channel of the YCbCr space or the V channel of the HSV space.

[0018] The optical flow method is used to calculate the horizontal and vertical displacements of all pixels in each frame of the image based on the extracted luminance component signals, and the displacement F of the pixels is calculated according to the following formula. (x,y) That is, the characteristics of change:

[0019]

[0020] In the formula, F x (i), F y (i) represents the displacement of the pixel with coordinates (x, y) from the i-th frame to the (i+1)-th frame in the x and y directions, respectively, and N is the number of frames in the video image;

[0021] The facial change feature map is obtained by arranging the pixels according to their positions.

[0022] Furthermore, the method for extracting facial change features from the enhanced image includes:

[0023] High-pass filtering is applied to each frame of the image, as shown below:

[0024] PH =PP GS

[0025] In the formula, P is the original image, P GS For the image after Gaussian filtering, P H This is the image after high-pass filtering;

[0026] Calculate the change in pixel value of each pixel in the image after high-pass filtering using the following formula:

[0027]

[0028] In the formula, V (x,y) (i) represents the pixel value of the pixel at coordinates (x, y) in the i-th frame of the image, Δv (x,y) Where N is the change in pixel value, and N is the number of frames in the video image;

[0029] The displacement direction of pixels in each frame of the image is calculated using the following formula:

[0030] θ (x,y) (i)=tan -1 F y (i) / F x (i)

[0031] In the formula, F x F y Let θ represent the displacements of the pixel with coordinates (x, y) from the (i-1)th frame to the i-th frame in the x and y directions, respectively. (x,y) (i) is the angle between the displacement direction and the x-axis, in degrees;

[0032] The change in the displacement direction of pixels in an image is calculated using the following formula:

[0033]

[0034] In the formula, Δθ (x,y) This represents the change in the direction of pixel displacement.

[0035] For Δv (x,y) and Δθ (x,y) Facial change features are obtained by fusion.

[0036] Furthermore, the method for identifying whether a face is real or fake based on extracted facial change features includes:

[0037] A classifier model is constructed based on a convolutional neural network, which takes facial change features as input and outputs face categories, including real faces and fake faces.

[0038] The model was trained using the attack dataset;

[0039] The facial change features extracted from face video images are input into the trained model, and the output is the face category.

[0040] Secondly, the present invention provides a face recognition device based on short videos, comprising:

[0041] The face detection module is used to acquire short videos containing target faces and perform face detection to obtain face video images;

[0042] The image enhancement module is used to enhance the image of facial change regions based on bandpass filtering of the facial video image;

[0043] The feature extraction module is used to extract facial change features from the enhanced image and identify whether it is a real face or a fake face based on the extracted facial change features.

[0044] The identity recognition module is used to perform identity recognition by extracting features from the face video image and matching them with the face features stored in the database if the face is a real face.

[0045] Furthermore, the method for enhancing the image of facial change regions includes:

[0046] Each frame of the image is decomposed using the Laplacian pyramid to obtain images at different scales;

[0047] Bandpass filtering is performed on the image at each scale, and the frequency signal within the filter's passband is output.

[0048] The pixel values ​​of the filtered image are multiplied by a magnification factor, which is different for each scale.

[0049] After the magnified image pyramid is restored to a single image, it is added to the image before the decomposition, and the added images are combined to form a video.

[0050] Furthermore, the method for extracting facial change features from the enhanced image includes:

[0051] Convert the RGB space image to the YCbCr or HSV space image, and extract the luminance component from the Y channel of the YCbCr space or the V channel of the HSV space.

[0052] The optical flow method is used to calculate the horizontal and vertical displacements of all pixels in each frame of the image based on the extracted luminance component signals, and the displacement F of the pixels is calculated according to the following formula. (x,y) That is, the characteristics of change:

[0053]

[0054] In the formula, F x(i), F y (i) represents the displacement of the pixel with coordinates (x, y) from the i-th frame to the (i+1)-th frame in the x and y directions, respectively, and N is the number of frames in the video image;

[0055] The facial change feature map is obtained by arranging the pixels according to their positions.

[0056] Furthermore, the method for extracting facial change features from the enhanced image includes:

[0057] High-pass filtering is applied to each frame of the image, as shown below:

[0058] P H =PP GS

[0059] In the formula, P is the original image, P GS For the image after Gaussian filtering, P H This is the image after high-pass filtering;

[0060] Calculate the change in pixel value of each pixel in the image after high-pass filtering using the following formula:

[0061]

[0062] In the formula, V (x,y) (i) represents the pixel value of the pixel at coordinates (x, y) in the i-th frame of the image, Δv (x,y) Where N is the change in pixel value, and N is the number of frames in the video image;

[0063] The displacement direction of pixels in each frame of the image is calculated using the following formula:

[0064] θ (x,y) (i)=tan -1 F y (i) / F x (i)

[0065] In the formula, F x F y Let θ represent the displacements of the pixel with coordinates (x, y) from the (i-1)th frame to the i-th frame in the x and y directions, respectively. (x,y) (i) is the angle between the displacement direction and the x-axis, in degrees;

[0066] The change in the displacement direction of pixels in an image is calculated using the following formula:

[0067]

[0068] In the formula, Δθ (x,y) This represents the change in the direction of pixel displacement.

[0069] For Δv(x,y) and Δθ (x,y) Facial change features are obtained by fusion.

[0070] Furthermore, the method for identifying whether a face is real or fake based on extracted facial change features includes:

[0071] A classifier model is constructed based on a convolutional neural network, which takes facial change features as input and outputs face categories, including real faces and fake faces.

[0072] The model was trained using the attack dataset;

[0073] The facial change features extracted from face video images are input into the trained model, and the output is the face category.

[0074] Compared with the prior art, the present invention has the following beneficial effects.

[0075] This invention acquires a short video containing a target face and performs face detection to obtain a face video image. It then enhances the facial change region image by applying bandpass filtering to the face video image. Facial change features are extracted from the enhanced image, and the extracted features are used to identify whether the face is real or fake. Finally, identity recognition is performed based on the real face video image, thus achieving face recognition based on short videos. This invention effectively extracts subtle facial change features by enhancing the facial change region image; and by detecting real and fake faces based on these extracted features, it can effectively detect face recognition attacks (such as photo attacks, video replay attacks, and model attacks), improving the accuracy and reliability of face recognition. Attached Figure Description

[0076] Figure 1 This is a flowchart of a face recognition method based on short videos according to an embodiment of the present invention.

[0077] Figure 2 This is a schematic diagram of face recognition based on face authenticity detection, according to another embodiment of the present invention.

[0078] Figure 3 A flowchart illustrating the process of facial feature enhancement.

[0079] Figure 4 This is a block diagram of a smart meter metering accuracy detection device based on big data, according to an embodiment of the present invention. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of this invention clearer and more understandable, the invention will be further described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0081] Figure 1 This is a flowchart of a face recognition method based on short videos according to an embodiment of the present invention. The method includes the following steps:

[0082] Step 101: Obtain a short video containing the target face and perform face detection to obtain a face video image;

[0083] Step 102: Enhance the image of the facial change region based on bandpass filtering of the face video image;

[0084] Step 103: Extract facial change features from the enhanced image and identify whether it is a real face or a fake face based on the extracted facial change features.

[0085] Step 104: If it is a real human face, perform feature extraction on the face video image and match it with the face features stored in the database to identify the person.

[0086] In this embodiment, step 101 is mainly used to obtain a face video image by performing face detection. First, a short video clip captured by a camera is acquired, then face detection is performed on the short video image, and images that do not contain faces are deleted to obtain a video image that includes face images. There are many methods for face detection, and this embodiment does not limit the specific face detection method.

[0087] In this embodiment, steps 102 and 103 are mainly used to perform face authenticity detection, that is, to identify real faces and fake faces. If it is a real face, further identity verification is performed; if it is a fake face, identity verification is not performed, and a face recognition failure result is directly output. Figure 2As shown, when using fake faces for attacks, the image clarity, printer quality, or the imaging device (such as the screen) can affect detail or introduce additional effects like moiré patterns. Therefore, methods for detecting fake faces based on image texture analysis have been proposed, utilizing the differences in facial texture features between real and fake faces. First, the image is divided into blocks, then multi-scale LBP (Local Binary Pattern) features are extracted from different regions, and finally, SVM (Support Vector Machine) is used for classification. Different color spaces contain different information; texture features can be extracted using CoLBP and LBQ in YCbCr and HSV color spaces for detection. Alternatively, parameters such as mean squared error and signal-to-noise ratio can be used to evaluate image quality, and the authenticity can be determined based on the evaluation results. Methods involving texture analysis and image quality detection from a single image are time-efficient and do not rely on additional equipment. However, with improvements in imaging and printing quality, the difference between real faces and attack samples is decreasing, and the detection capability is gradually declining. Since many subtle facial changes that are difficult for humans to detect often contain very important feature information, this embodiment proposes a method for detecting the authenticity of faces based on facial change features. When people perform face recognition, they are generally static; therefore, real face images mostly show local changes. However, fake faces generally exhibit overall movement (photo attacks and model attacks clearly show overall movement, and even video replay attacks also exhibit overall movement due to secondary imaging). Therefore, the authenticity of faces can be detected based on local change features.

[0088] Since facial changes are generally not obvious, to effectively extract these facial change features, this embodiment first enhances the image of the facial change area in step 102, and then extracts facial change features from the enhanced image in step 103. Facial changes affect the frequency of the facial image; therefore, bandpass filtering can be applied to the original image to output the frequency components after facial changes. The filtered output signal is then amplified to enhance the facial change signal. A specific enhancement method will be provided in later embodiments. Step 103 extracts facial change features from the enhanced image obtained in step 102. Facial changes are mainly manifested in the magnitude and speed of pixel displacement; therefore, facial change feature extraction mainly involves obtaining data such as the displacement amount, direction, or speed of pixels. This embodiment does not limit the specific facial change feature extraction method; two specific implementation methods will be provided in later embodiments. Step 103 also performs face authenticity detection based on the extracted facial change features to obtain face categories: real faces and fake faces. By constructing a binary classifier and inputting the extracted facial change features into the trained classifier, the face category of the video image can be obtained.

[0089] In this embodiment, step 104 is mainly used for identity verification. A necessary condition for identity verification is that the face category output by the face authenticity detection is a real face; or, if the output face category is a fake face, then identity verification is unnecessary, and the recognition fails directly. Identity verification is achieved by matching the extracted facial features with the facial features of legitimate individuals stored in the database. If the match is successful, the identity is legitimate, and face recognition is successful; if the match fails, the identity is illegitimate, and face recognition fails. Figure 2 As shown.

[0090] As an optional embodiment, the method for enhancing the image of facial change regions includes:

[0091] Each frame of the image is decomposed using the Laplacian pyramid to obtain images at different scales;

[0092] Bandpass filtering is performed on the image at each scale, and the frequency signal within the filter's passband is output.

[0093] The pixel values ​​of the filtered image are multiplied by a magnification factor, which is different for each scale.

[0094] After the magnified image pyramid is restored to a single image, it is added to the image before the decomposition, and the added images are combined to form a video.

[0095] This embodiment provides a technical solution for enhancing images of facial change regions. The flowchart for image enhancement is as follows: Figure 3 As shown. First, spatial decomposition is performed, using Laplacian pyramid decomposition to process each frame of the image, resulting in images at different scales. Next, bandpass filtering is applied to each scale of the image. The passband of the bandpass filter is the frequency range of facial changes, typically 0.75–2.0 Hz. The purpose of bandpass filtering is to filter out facial change feature image signals from the frequency domain, while filtering out non-facial change feature image signals. Then, the filtered facial change feature image signals are amplified, that is, the pixel values ​​of the image are multiplied by a magnification factor. This can be achieved by unifying the image size to a uniform size, such as 64*64, then multiplying the pixel values ​​by the magnification factor, and then restoring it to the original size. The magnification factor should not be too low or too high; too low a factor will not achieve the desired enhancement effect, while too high a factor will cause distortion in the amplified video image. The magnification factor is generally determined through multiple experiments, such as setting it to 15 times. Finally, video reconstruction is performed. The amplified image is first restored using a pyramid, then added to the image before pyramid decomposition, and the combined image is used to assemble the video.

[0096] As an optional embodiment, the method for extracting facial change features from the enhanced image includes:

[0097] Convert the RGB space image to the YCbCr or HSV space image, and extract the luminance component from the Y channel of the YCbCr space or the V channel of the HSV space.

[0098] The optical flow method is used to calculate the horizontal and vertical displacements of all pixels in each frame of the image based on the extracted luminance component signals, and the displacement F of the pixels is calculated according to the following formula. (x,y) That is, the characteristics of change:

[0099]

[0100] In the formula, F x (i), F y (i) represents the displacement of the pixel with coordinates (x, y) from the i-th frame to the (i+1)-th frame in the x and y directions, respectively, and N is the number of frames in the video image;

[0101] The facial change feature map is obtained by arranging the pixels according to their positions.

[0102] This embodiment provides a technical solution for facial change feature extraction. As mentioned earlier, facial changes cause changes in pixel positions. Therefore, the displacement or movement speed of the same pixel in different frames can be used as facial change features. This embodiment uses the displacement magnitude of the same pixel in two adjacent frames as the change feature. The pixel displacement is calculated as follows: first, the horizontal and vertical displacements of the pixel in two adjacent frames are calculated using optical flow; then, the actual displacement is calculated using the Pythagorean theorem; and finally, the average displacement is calculated. Optical flow is a mature existing technology, and the specific solution steps will not be described in detail here. Since the image needs to be converted to grayscale when using optical flow, but the motion state information is mainly reflected by changes in image brightness, the original RGB color space image should be converted to another color space before displacement calculation, and then the image brightness information should be extracted. Commonly used color spaces are YCbCr and HSV. In the YCbCr color space, Y represents brightness, Cb represents the blue chromaticity component, and Cr represents the red chromaticity component; in the HSV color space, H represents hue, S represents saturation, and V represents brightness. The luminance component can be extracted from the Y channel of the converted YCbCr space or the V channel of the HSV space.

[0103] As an optional embodiment, the method for extracting facial change features from the enhanced image includes:

[0104] High-pass filtering is applied to each frame of the image, as shown below:

[0105] P H =PP GS

[0106] In the formula, P is the original image, P GS For the image after Gaussian filtering, P H This is the image after high-pass filtering;

[0107] Calculate the change in pixel values ​​of pixels in the image after high-pass filtering using the following formula:

[0108]

[0109] In the formula, V (x,y) (i) represents the pixel value of the pixel at coordinates (x, y) in the i-th frame image, Δv (x,y) N represents the change in pixel value, and N is the number of frames in the image.

[0110] The displacement direction of pixels in each frame of the image is calculated using the following formula:

[0111] θ (x,y) (i)=tan -1 F y (i) / F x (i)

[0112] In the formula, F x F y Let θ represent the displacements of the pixel with coordinates (x, y) from the (i-1)th frame to the i-th frame in the x and y directions, respectively. (x,y) (i) is the angle between the displacement direction and the x-axis, in degrees;

[0113] The change in the displacement direction of pixels in an image is calculated using the following formula:

[0114]

[0115] In the formula, Δθ (x,y) This represents the change in the direction of pixel displacement.

[0116] For Δv (x,y) and Δθ (x,y) Facial change features are obtained by fusion.

[0117] This embodiment presents another technical solution for facial change feature extraction. Unlike the previous embodiment, this embodiment uses the fusion of changes in pixel values ​​and changes in pixel displacement direction as facial change features. Therefore, it is necessary to obtain these two changes separately first. Since the local image frequency of facial changes is high, this embodiment first performs high-pass filtering on the video image to effectively obtain the change features. This embodiment achieves high-pass filtering by first performing low-pass filtering on the video image using a Gaussian filter, and then subtracting the Gaussian-filtered image from the original image. Then, for the high-pass-filtered image, the change in pixel value of each pixel relative to the average pixel value in each frame is calculated, and the average change value is obtained to obtain the change in pixel value for each pixel; next, the change in displacement direction of each pixel relative to the average displacement direction in each frame is calculated, and the average change value is obtained to obtain the change in displacement direction for each pixel. The angle between the displacement direction and the horizontal direction is obtained by calculating the arctangent of the ratio of the vertical displacement to the horizontal displacement. The vertical and horizontal displacements are still calculated using the optical flow method. Finally, the two changes are fused to obtain the facial change features.

[0118] As an optional embodiment, the method for identifying whether a face is real or fake based on extracted facial change features includes:

[0119] A classifier model is constructed based on a convolutional neural network, which takes facial change features as input and outputs face categories, including real faces and fake faces.

[0120] The model was trained using the attack dataset;

[0121] The facial change features extracted from face video images are input into the trained model, and the output is the face category.

[0122] This embodiment presents a technical solution for identifying face categories based on extracted facial change features. There are two face categories in this embodiment: real faces and fake faces. Therefore, this embodiment constructs a binary classifier to output the face category. This embodiment uses a deep convolutional neural network to construct the model framework for the binary classifier. VGG16 can be used as the base network of the model. VGG16 is widely used in image recognition and image classification due to its advantages such as good network performance and strong portability. The input of VGG16 is a three-channel RGB image with a size of 224*224. Each convolutional group includes 1-4 convolutional layers, followed by a max-pooling layer, for a total of five convolutional blocks consisting of one convolutional group and one pooling layer. Three fully connected layers are connected after the convolutional layers, and a softmax layer is used to output the final result. The network uses two 3*3 convolutional kernels instead of 5*5 convolutional kernels, significantly reducing the number of parameters and enhancing the model's nonlinearity and expressive power. Therefore, it features small convolutional kernels, few parameters, and a large receptive field. The classifier model parameters are determined through model training, thus requiring a high-quality dataset with attack methods. Several public datasets are currently available, such as NUAA, Print-attack, and Replay-attack. Alternatively, you can create your own dataset as needed. Once the model is trained, it can make predictions based on the input facial variation features.

[0123] Figure 4 This is a schematic diagram illustrating the composition of a face recognition device based on short videos according to an embodiment of the present invention. The device includes:

[0124] The face detection module 11 is used to acquire a short video containing the target face and perform face detection to obtain a face video image;

[0125] Image enhancement module 12 is used to enhance the image of facial change regions based on bandpass filtering of the face video image;

[0126] The feature extraction module 13 is used to extract facial change features from the enhanced image and identify whether it is a real face or a fake face based on the extracted facial change features.

[0127] The identity recognition module 14 is used to perform identity recognition by extracting features from the face video image and matching them with the face features stored in the database if the face is a real face.

[0128] The apparatus of this embodiment can be used to perform Figure 1 The technical solutions of the illustrated method embodiments are similar in principle and technical effect, and will not be described again here. The same applies to the subsequent embodiments, which will not be elaborated upon further.

[0129] As an optional embodiment, the method for enhancing the image of facial change regions includes:

[0130] Each frame of the image is decomposed using the Laplacian pyramid to obtain images at different scales;

[0131] Bandpass filtering is performed on the image at each scale, and the frequency signal within the filter's passband is output.

[0132] The pixel values ​​of the filtered image are multiplied by a magnification factor, which is different for each scale.

[0133] After the magnified image pyramid is restored to a single image, it is added to the image before the decomposition, and the added images are combined to form a video.

[0134] As an optional embodiment, the method for extracting facial change features from the enhanced image includes:

[0135] Convert the RGB space image to the YCbCr or HSV space image, and extract the luminance component from the Y channel of the YCbCr space or the V channel of the HSV space.

[0136] The optical flow method is used to calculate the horizontal and vertical displacements of all pixels in each frame of the image based on the extracted luminance component signals, and the displacement F of the pixels is calculated according to the following formula. (x,y) That is, the characteristics of change:

[0137]

[0138] In the formula, F x (i), F y (i) represents the displacement of the pixel with coordinates (x, y) from the i-th frame to the (i+1)-th frame in the x and y directions, respectively, and N is the number of frames in the video image;

[0139] The facial change feature map is obtained by arranging the pixels according to their positions.

[0140] As an optional embodiment, the method for extracting facial change features from the enhanced image includes:

[0141] High-pass filtering is applied to each frame of the image, as shown below:

[0142] P H =PP GS

[0143] In the formula, P is the original image, P GS For the image after Gaussian filtering, P H This is the image after high-pass filtering;

[0144] Calculate the change in pixel values ​​of pixels in the image after high-pass filtering using the following formula:

[0145]

[0146] In the formula, V (x,y) (i) represents the pixel value of the pixel at coordinates (x, y) in the i-th frame image, Δv (x,y) Where N is the change in pixel value, and N is the number of frames in the video image;

[0147] The displacement direction of pixels in each frame of the image is calculated using the following formula:

[0148] θ (x,y) (i)=tan -1 F y (i) / F x (i)

[0149] In the formula, F x F y Let θ represent the displacements of the pixel with coordinates (x, y) from the (i-1)th frame to the i-th frame in the x and y directions, respectively. (x,y) (i) is the angle between the displacement direction and the x-axis, in degrees;

[0150] The change in the displacement direction of pixels in an image is calculated using the following formula:

[0151]

[0152] In the formula, Δθ (x,y) This represents the change in the direction of pixel displacement.

[0153] For Δv (x,y) and Δθ (x,y) Facial change features are obtained by fusion.

[0154] As an optional embodiment, the method for identifying whether a face is real or fake based on extracted facial change features includes:

[0155] A classifier model is constructed based on a convolutional neural network, which takes facial change features as input and outputs face categories, including real faces and fake faces.

[0156] The model was trained using the attack dataset;

[0157] The facial change features extracted from face video images are input into the trained model, and the output is the face category.

[0158] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A face recognition method based on short videos, characterized in that, Includes the following steps: Step 1: Obtain a short video containing the target face and perform face detection to obtain a face video image; Step 2: Enhance the facial change region image by applying bandpass filtering to the face video image; the facial change region image has the characteristic that the same pixel point is in different positions in different frame images due to local facial movement; Step 3: Extract facial change features from the enhanced image and identify whether it is a real face or a fake face based on the extracted facial change features; the method for extracting facial change features from the enhanced image includes: High-pass filtering is applied to each frame of the image, as shown below: In the formula, For the original image, The image is after Gaussian filtering. This is the image after high-pass filtering; Calculate the change in pixel value of each pixel in the image after high-pass filtering using the following formula: In the formula, For the first i The coordinates in the frame image are The pixel value of the pixel. The change in pixel value N The number of frames in the video image; The displacement direction of pixels in each frame of the image is calculated using the following formula: In the formula, , The coordinates are respectively The pixel from the first i -1 frame image to the i Frame image in , Displacement in direction, The direction of displacement and The included angle of the axes, in degrees; The change in the displacement direction of pixels in an image is calculated using the following formula: In the formula, This represents the change in the direction of pixel displacement. right and Facial change features are obtained by fusion; Step 4: If it is a real human face, the face video image is used to extract features and then matched with the facial features stored in the database for identity recognition.

2. The face recognition method based on short videos according to claim 1, characterized in that, The method for enhancing images of facial change regions includes: Each frame of the image is decomposed using the Laplacian pyramid to obtain images at different scales; Bandpass filtering is performed on the image at each scale, and the frequency signal within the filter's passband is output. The pixel values ​​of the filtered image are multiplied by a magnification factor, which is different for each scale. After the magnified image pyramid is restored to a single image, it is added to the image before the decomposition, and the added images are combined to form a video.

3. The face recognition method based on short videos according to claim 2, characterized in that, The method for extracting facial change features from the enhanced image includes: Convert the RGB space image to the YCbCr or HSV space image, and extract the luminance component from the Y channel of the YCbCr space or the V channel of the HSV space. The optical flow method is used to calculate the horizontal and vertical displacements of all pixels in each frame of the image based on the extracted luminance component signals, and the displacement of each pixel is calculated using the following formula. That is, the characteristics of change: In the formula, , The coordinates are respectively The pixel from the first i Frame image to the i +1 frame image in , Displacement in direction, N The number of frames in the video image; The facial change feature map is obtained by arranging the pixels according to their positions.

4. The face recognition method based on short videos according to claim 1, characterized in that, The method for identifying whether a face is real or fake based on extracted facial change features includes: A classifier model is constructed based on a convolutional neural network, which takes facial change features as input and outputs face categories, including real faces and fake faces. The model was trained using the attack dataset; The facial change features extracted from face video images are input into the trained model, and the output is the face category.

5. A facial recognition device based on short videos, characterized in that, include: The face detection module is used to acquire short videos containing target faces and perform face detection to obtain face video images; The image enhancement module is used to enhance the facial change region image based on bandpass filtering of the face video image; the facial change region image has the characteristic that the same pixel point is in different positions in different frame images due to local facial movement; The feature extraction module is used to extract facial change features from the enhanced image and identify whether the extracted facial change features represent a real or fake face. The method for extracting facial change features from the enhanced image includes: High-pass filtering is applied to each frame of the image, as shown below: In the formula, For the original image, The image is after Gaussian filtering. This is the image after high-pass filtering; Calculate the change in pixel value of each pixel in the image after high-pass filtering using the following formula: In the formula, For the first i The coordinates in the frame image are The pixel value of the pixel. The change in pixel value N The number of frames in the video image; The displacement direction of pixels in each frame of the image is calculated using the following formula: In the formula, , The coordinates are respectively The pixel from the first i -1 frame image to the i Frame image in , Displacement in direction, The direction of displacement and The included angle of the axes, in degrees; The change in the displacement direction of pixels in an image is calculated using the following formula: In the formula, This represents the change in the direction of pixel displacement. right and Facial change features are obtained by fusion; The identity recognition module is used to perform identity recognition by extracting features from the face video image and matching them with the face features stored in the database if the face is a real face.

6. The face recognition device based on short video according to claim 5, characterized in that, The method for enhancing images of facial change regions includes: Each frame of the image is decomposed using the Laplacian pyramid to obtain images at different scales; Bandpass filtering is performed on the image at each scale, and the frequency signal within the filter's passband is output. The pixel values ​​of the filtered image are multiplied by a magnification factor, which is different for each scale. After the magnified image pyramid is restored to a single image, it is added to the image before the decomposition, and the added images are combined to form a video.

7. The face recognition device based on short video according to claim 6, characterized in that, The method for extracting facial change features from the enhanced image includes: Convert the RGB space image to the YCbCr or HSV space image, and extract the luminance component from the Y channel of the YCbCr space or the V channel of the HSV space. The optical flow method is used to calculate the horizontal and vertical displacements of all pixels in each frame of the image based on the extracted luminance component signals, and the displacement of each pixel is calculated using the following formula. That is, the characteristics of change: In the formula, , The coordinates are respectively The pixel from the first i Frame image to the i +1 frame image in , Displacement in direction, N The number of frames in the video image; The facial change feature map is obtained by arranging the pixels according to their positions.

8. The face recognition device based on short video according to claim 5, characterized in that, The method for identifying whether a face is real or fake based on extracted facial change features includes: A classifier model is constructed based on a convolutional neural network, which takes facial change features as input and outputs face categories, including real faces and fake faces. The model was trained using the attack dataset; The facial change features extracted from face video images are input into the trained model, and the output is the face category.

Images