Face forgery detection method and device, equipment, storage medium and computer program product

Abstract

The invention relates to the technical field of videos, and discloses a face forgery detection method, device and equipment, a storage medium and a computer program product, and the method comprises the steps: extracting a frequency awareness decomposition feature and a local frequency statistical feature of a to-be-detected face image, and performing anti-counterfeiting detection on the to-be-detected face image based on the frequency awareness decomposition feature, the local frequency statistical feature and the target bidirectional long-short term memory network model to obtain an anti-counterfeiting detection result of the to-be-detected face image, the target bidirectional long-short-term memory network model is a bidirectional attention long-short-term memory network model combined with spatial frequency two dimensions; due to the fact that the double-flow branch network is used, the dynamic difference between the video frames is explored, richer time sequence feature information is extracted in combination with the two-way long-short-term memory network model with the attention mechanism, effective detection of the face counterfeit video is achieved, and the detection accuracy is improved. Therefore, the problem of face anti-counterfeiting detection in a complex natural scene is solved.

Description

Technical Field

[0001] The present application relates to the field of video technology, and in particular to a method, device, equipment, storage medium and computer program product for detecting face forgery. Background Art

[0002] At present, some face forgery detection scenarios are in complex detection environments. Attackers may use traditional non-live attack methods or use advanced AI (Artificial Intelligence) face-changing technology. Related face forgery attack detection methods are mainly targeted at specific fields and lack attention to complex scenarios, resulting in low accuracy. Summary of the invention

[0003] The main purpose of this application is to provide a face forgery detection method, device, equipment, storage medium and computer program product, aiming to solve the technical problem that related face forgery attack detection methods are mainly targeted at specific fields and lack attention to complex scenes, resulting in low accuracy.

[0004] To achieve the above objectives, the present application provides a method for detecting forged faces, the method comprising:

[0005] Extract frequency-aware decomposition features and local frequency statistical features of the face image to be detected;

[0006] Based on the frequency awareness decomposition features, the local frequency statistical features and the target bidirectional long short-term memory network model, anti-counterfeiting detection is performed on the face image to be detected to obtain an anti-counterfeiting detection result of the face image to be detected, wherein the target bidirectional long short-term memory network model is a bidirectional attention long short-term memory network model that combines the dual dimensions of space and frequency.

[0007] Optionally, the anti-counterfeiting detection is performed on the face image to be detected based on the frequency awareness decomposition feature, the local frequency statistical feature and the target bidirectional long short-term memory network model to obtain the anti-counterfeiting detection result of the face image to be detected, including:

[0008] Fusing the frequency awareness decomposition feature with the local frequency statistical feature to obtain a fused feature;

[0009] Inputting the face image to be detected and the fusion features into a target bidirectional long short-term memory network model to obtain temporal feature information;

[0010] The temporal feature information is fused with the fusion feature to obtain a confidence level, wherein the confidence level is used to characterize an anti-counterfeiting detection result of a face image to be detected.

[0011] Optionally, the step of extracting frequency-aware decomposition features and local frequency statistical features of the face image to be detected includes:

[0012] Performing discrete cosine transformation on the face image to be detected, and extracting features of the changed face image through a filter to obtain frequency-aware decomposition features of the face image to be detected, wherein the filter is a weighted combination of multiple binary classification filters and multiple learnable filters, and the multiple binary classification filters divide the frequency of the face image to be detected into multiple frequency bands;

[0013] The local frequency domain statistical information of the face image to be detected is extracted by sliding window discrete cosine transform, and the local frequency statistical features of the face image to be detected are generated according to the local frequency domain statistical information.

[0014] Optionally, before extracting the frequency-aware decomposition features and local frequency statistical features of the face image to be detected, the method further includes:

[0015] Acquire video frame data to be detected, and perform face image segmentation on the video frame data to obtain a face image to be detected;

[0016] Performing liveness detection on the face image to be detected by using a command liveness detection mode and a colorful silent liveness detection mode to obtain a liveness detection result of the face image to be detected;

[0017] Accordingly, the step of extracting the frequency-aware decomposition features and local frequency statistical features of the face image to be detected includes:

[0018] When the liveness detection result is detection passed, frequency awareness decomposition features and local frequency statistical features of the face image to be detected are extracted.

[0019] Optionally, performing liveness detection on the face image to be detected by using a command liveness detection mode and a colorful silent liveness detection mode to obtain a liveness detection result of the face image to be detected includes:

[0020] Identifying different facial movements in the face image to be detected to obtain a command liveness detection result;

[0021] Detect several consecutive frames of face images to be detected to obtain colorful silent liveness detection results;

[0022] The liveness detection result of the face image to be detected is generated according to the instruction liveness detection result and the colorful silent liveness detection result.

[0023] Optionally, the acquiring of the video frame data to be detected and performing face image segmentation on the video frame data to obtain the face image to be detected includes:

[0024] Acquire video frame data to be detected, and perform face image segmentation on the video frame data through a target detection model to obtain a segmented image;

[0025] Obtaining the position of human eyes in the segmented image, and calculating the rotation angle of the face image according to the position of human eyes;

[0026] The segmented image is rotated according to the rotation angle of the face image to obtain a face image to be detected.

[0027] In addition, to achieve the above-mentioned purpose, the present application also proposes a face forgery detection device, the face forgery detection device comprising:

[0028] A feature extraction module, used to extract frequency-aware decomposition features and local frequency statistical features of the face image to be detected;

[0029] The anti-counterfeiting detection module is used to perform anti-counterfeiting detection on the face image to be detected based on the frequency awareness decomposition features, the local frequency statistical features and the target bidirectional long short-term memory network model to obtain the anti-counterfeiting detection result of the face image to be detected, wherein the target bidirectional long short-term memory network model is a bidirectional attention long short-term memory network model that combines the dual dimensions of space and frequency.

[0030] In addition, to achieve the above-mentioned purpose, the present application also proposes a face forgery detection device, which includes a memory, a processor, and a face forgery detection program stored in the memory and executable on the processor, wherein the face forgery detection program is configured to implement the face forgery detection method described above.

[0031] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, on which a face forgery detection program is stored, and when the face forgery detection program is executed by a processor, the face forgery detection method as described above is implemented.

[0032] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a face forgery detection program, and when the face forgery detection program is executed by a processor, it implements the face forgery detection method described above.

[0033] One or more technical solutions proposed in this application have at least the following technical effects:

[0034] In the present application, a method for extracting frequency-aware decomposition features and local frequency statistical features of a face image to be detected is disclosed, and anti-counterfeiting detection is performed on the face image to be detected based on the frequency-aware decomposition features, the local frequency statistical features and a target bidirectional long short-term memory network model to obtain an anti-counterfeiting detection result of the face image to be detected, wherein the target bidirectional long short-term memory network model is a bidirectional attention long short-term memory network model that combines the spatial frequency dual dimensions; since the present application uses a dual-stream branch network to explore the dynamic differences between video frames, one branch is used to extract dynamic timing inconsistencies in multi-frame videos, and the other branch is used to amplify forged details, and combined with the bidirectional long short-term memory network model with an attention mechanism, richer timing feature information is extracted, thereby achieving effective detection of face forgery videos, thereby solving the problem of face anti-counterfeiting detection in complex natural scenes and improving detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0037] Figure 1 This is a flow chart of the first embodiment of the face forgery detection method of the present application;

[0038] Figure 2 This is a flow chart of the second embodiment of the face forgery detection method of the present application;

[0039] Figure 3 This is a model overall structure diagram of an embodiment of the face forgery detection method of the present application;

[0040] Figure 4 This is a BLA network structure combining spatial and frequency dual dimensions according to an embodiment of the face forgery detection method of the present applicant;

[0041] Figure 5 This is a flow chart of the third embodiment of the face forgery detection method of the present application;

[0042] Figure 6 A schematic diagram of a right triangle with a tilted face according to an embodiment of the applicant's face forgery detection method;

[0043] Figure 7 This is an overall flow chart of an embodiment of the face forgery detection method of the present application;

[0044] Figure 8 This is a schematic diagram of the module structure of the face forgery detection device according to an embodiment of the present application;

[0045] Fig. 9 Schematic diagram of the device structure of the hardware operating environment involved in the face forgery detection method in the embodiment of the present application.

[0046] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0048] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0049] "5G e-Signature" is a digital trust product that focuses on the development and innovation of the digital economy and adapts to future needs. The product supports online signing anytime and anywhere, provides safe and reliable electronic signature and electronic seal services, realizes paperless signing and transparent processes, thereby significantly improving work efficiency, reducing operating costs, and saving resources such as paper, manpower and time, helping to achieve the dual-carbon strategy and achieve the goal of reducing costs and increasing efficiency. In addition, the product is committed to fully empowering governments and enterprises through intelligent means to promote their efficient collaboration and high-quality development. The video signing technical solution requires identity authentication for online signatories, so it puts forward higher requirements for the identity security and data security of signatories, and urgently needs to have the ability to defend against face forgery attacks.

[0050] According to research, current face forgery attacks can be roughly divided into two types: traditional forgery and AI (Artificial intelligence) face-changing.

[0051] Traditional forgery methods include image attacks (using printed photos to attack), video attacks (playing pre-recorded videos) and mask attacks (using forged masks to attack). Among them, non-live attacks such as images and masks pose a higher threat and are usually detected using algorithms such as colorful liveness or interactive liveness.

[0052] AI face-swapping is difficult to detect. This technology uses deep learning-based image processing technology to accurately replace the target person's facial information with the replacement's facial information. As the target person's facial expression changes in the video, the replacement's facial expression also changes synchronously, making the video look the same and difficult to detect forgery.

[0053] At present, AI face-changing detection technology mainly includes two types of methods: image forgery detection method and video forgery detection method.

[0054] 1. Image forgery detection method:

[0055] The image forgery detection method is a technology used to detect the image or video frame to be detected. According to its principle, the technology includes two methods, one is the image forgery detection method based on the spatial domain, and the other is the image forgery detection method based on the frequency domain.

[0056] 1.1. Image forgery detection method based on spatial domain:

[0057] Early image classification models such as deep separable convolutional networks and efficient convolutional neural networks performed well in image forgery detection. However, these models are not specifically designed for face forgery detection, so attempts were made to add texture enhancement modules, attention generation modules, and bilinear attention pooling modules to guide the model to pay more attention to facial features, such as texture details. However, it has been verified that these methods cause the model to overfit to specific forgery methods, and their detection performance is poor for forgery methods that have not been trained.

[0058] To further improve the generalization ability of the model, the following measures have been taken: On the one hand, real face images are processed to generate forged face images to solve the problem of overfitting the model to a specific forgery method. On the other hand, through paired self-consistency learning, the model is encouraged to learn more generalized feature representations. In addition, there are methods to learn more compact real face features by reconstructing the learning auxiliary model.

[0059] In general, current spatial domain-based detection methods have made some progress in texture details, self-consistency learning, and reconstruction learning. However, these methods still face challenges, such as adapting to new forgery methods and detecting whole-face synthesis methods.

[0060] 1.2. Image forgery detection method based on frequency domain:

[0061] After images and videos transmitted over the Internet are compressed multiple times, traces of forgery are almost invisible. Therefore, many researchers turn to other fields such as frequency domain to obtain clues for forgery detection. Some methods, such as frequency-domain-aware decomposition module and local frequency information statistics module, use frequency domain information to capture forgery clues and show good detection performance on highly compressed images. Other methods, such as adaptive frequency feature generation module and single center loss, improve forgery detection performance by learning differential features in different frequency bands and increasing the difference between real and forged faces.

[0062] Frequency domain based methods show strong detection capabilities when dealing with highly compressed forged images. However, their performance degrades significantly when faced with unknown forgery methods.

[0063] 2. Video forgery detection method:

[0064] Existing forged videos are usually created by forging each frame of the original video and then splicing them together. Therefore, there are often inconsistencies in texture and local details between the previous and next frames of the forged video. Video forgery detection methods can use this temporal inconsistency to detect. According to the detection principle, these methods can be divided into time domain-based detection methods and multimodal-based detection methods.

[0065] 2.1. Detection method based on time domain:

[0066] In order to explore the dynamic differences between video frames, a two-stream branch network is proposed. One branch is used to extract dynamic temporal inconsistencies in multi-frame videos, and the other branch applies the LoG (Laplacian of Gaussian) operator to amplify the forged details. In addition, the researchers introduced the Deep SVDD (Deep Support Vector Data Description) loss function, which can enhance the intra-class compactness of real faces and improve the inter-class distinction between real and forged faces, thereby improving the generalization ability of the model.

[0067] The study found that 3D convolution with a temporal convolution kernel size of 1 can enhance the network's ability to express temporal information and help capture temporal inconsistencies in forged videos. This method shows excellent detection capabilities when facing unknown forgery methods. However, since temporal inconsistencies are easily disturbed by factors such as noise and compression, the robustness of such methods is relatively weak.

[0068] 2.2. Detection method based on multimodality:

[0069] By associating face videos and audio, the researchers proposed using residual networks to extract audio features and using channel separation convolutional networks to extract video features. They used a self-supervised approach to learn the natural correspondence between faces and audio in real videos, such as how facial expressions change with audio. This method showed excellent robustness and generalization performance on forgery detection datasets. However, this type of detection method requires pre-training tasks on large-scale datasets, so it has certain requirements for the support of a large amount of computing resources, which sets a high threshold for subsequent research work.

[0070] In summary, the face anti-counterfeiting detection method based on time domain cannot fully extract time series related information and is greatly affected by factors such as noise and compression. The face anti-counterfeiting detection method based on multimodality depends on aspects such as the size of the data set and computing power requirements. Therefore, this application proposes a deep learning face anti-counterfeiting detection method that comprehensively considers multi-dimensional portrait features such as frequency domain, spatial domain, and time series, in order to overcome the limitations of existing methods.

[0071] Therefore, in order to overcome the above-mentioned defects, the present application provides a solution, which includes: extracting frequency-aware decomposition features and local frequency statistical features of the face image to be detected, performing anti-counterfeiting detection on the face image to be detected based on the frequency-aware decomposition features, local frequency statistical features and a target bidirectional long short-term memory network model, and obtaining an anti-counterfeiting detection result of the face image to be detected, wherein the target bidirectional long short-term memory network model is a bidirectional attention long short-term memory network model that combines the two dimensions of space and frequency; since the present application uses a dual-stream branch network to explore the dynamic differences between video frames, one branch is used to extract dynamic timing inconsistencies in multi-frame videos, and the other branch is used to amplify forged details, and combined with the bidirectional long short-term memory network model with an attention mechanism to extract richer timing feature information, thereby achieving effective detection of face forgery videos, thereby solving the problem of face anti-counterfeiting detection in complex natural scenes and improving detection accuracy.

[0072] Based on this, the present application embodiment provides a face forgery detection method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the face forgery detection method of the present application.

[0073] In a first embodiment, the face forgery detection method includes:

[0074] Step S10: extracting frequency-aware decomposition features and local frequency statistical features of the face image to be detected.

[0075] It should be understood that the executor of this embodiment can be a face forgery detection device with data processing, network communication and program running functions, such as a computer, or other electronic devices that can achieve the same or similar functions, and this embodiment does not impose any restrictions on this.

[0076] It can be understood that FAD (Frequency-Aware Decomposition) is a frequency-domain-aware decomposition technology used to extract frequency-domain features (i.e., frequency-aware decomposition features (FAD features)) from images or videos. In specific implementations, DCT (Discrete Cosine Transform) or other similar transformation methods are used to capture information about images at different frequencies. Unlike the traditional method of manually designing fixed filters to extract frequency-domain features, FAD uses learnable filters that can adaptively decompose image frequency-domain features, thereby more accurately detecting forgery traces in different frequency band components.

[0077] LFS (Local Frequency Statistics) can be used to extract local frequency responses (i.e. local frequency statistics features (LFS features)) in images or videos through sliding window DCT or other local frequency domain analysis methods. LFS maintains translation invariance and local consistency and is more sensitive to details and local anomalies in the image.

[0078] Step S20: Perform anti-counterfeiting detection on the face image to be detected based on the frequency awareness decomposition features, the local frequency statistical features and the target bidirectional long short-term memory network model to obtain the anti-counterfeiting detection result of the face image to be detected, wherein the target bidirectional long short-term memory network model is a bidirectional attention long short-term memory network model that combines the dual dimensions of space and frequency.

[0079] It should be understood that in this embodiment, two frequency domain features, namely FAD features and LFS features, are first extracted from the original video sequence, and their features in the network are fused through the fusion module Fusion Box. Then, the model passes the original video sequence and the output of the Fusion Box as input into the bidirectional attention LSTM (Long Short-Term Memory) network structure BLA (Bi-LSTM with Attention, bidirectional attention long short-term memory network) that combines the dual dimensions of space and frequency, thereby extracting richer temporal feature information that integrates spatial frequency information from the video sample set data. Finally, the multi-dimensional temporal features extracted by the bidirectional LSTM are fused with the aforementioned frequency domain features to achieve effective detection of forged face videos (such as AI face-changing, etc.).

[0080] In a specific implementation, for example, the FAD feature is fused with the LFS feature to obtain the fused feature; the face image to be detected and the fused feature are input into the target bidirectional LSTM model to obtain time series feature information; the time series feature information is fused with the fused feature to obtain confidence, wherein the confidence is used to characterize the anti-counterfeiting detection result of the face image to be detected.

[0081] This embodiment uses a two-stream branch network to explore the dynamic differences between video frames. One branch is used to extract dynamic timing inconsistencies in multi-frame videos, and the other branch is used to amplify forgery details. It is combined with a bidirectional LSTM model with an attention mechanism to extract richer timing feature information, thereby achieving effective detection of forged face videos, and further solving the problem of face anti-counterfeiting detection in complex natural scenes and improving detection accuracy.

[0082] Reference Figure 2 , Figure 2 This is a flow chart of the second embodiment of the face forgery detection method of the present application, based on the above Figure 1 The first embodiment shown here proposes the second embodiment of the applicant's face forgery detection method.

[0083] In the second embodiment, the step S10 includes:

[0084] Step S101: Perform discrete cosine transform on the face image to be detected, and extract features of the changed face image through a filter to obtain frequency-aware decomposition features of the face image to be detected, wherein the filter is a weighted combination of multiple binary classification filters and multiple learnable filters, and the multiple binary classification filters divide the frequency of the face image to be detected into multiple frequency bands.

[0085] It should be understood that the traditional method uses manually designed fixed filters to extract frequency domain features, which cannot completely cover all image patterns. In this embodiment, the FAD feature uses a learnable filter to adaptively decompose the image frequency domain features, which can more accurately detect forgery traces in different frequency bands. The model contains multiple binary classification filters f base , which divides the frequency of the image into three frequency bands: low, medium, and high. Since fixed filters are difficult to adaptively capture the forgery patterns in the image, the model also contains three learnable filters f W , by combining the above two filters weightedly, the decomposed frequency domain features can be extracted.

[0086] After the image is input, the model first performs DCT transformation on it, then inputs it into the filter for feature extraction, and finally performs inverse DCT transformation to reconstruct the image.

[0087] Step S102: extracting local frequency domain statistical information of the face image to be detected by sliding window discrete cosine transform, and generating local frequency statistical features of the face image to be detected according to the local frequency domain statistical information.

[0088] It should be understood that the FAD feature extracts frequency domain features and transforms them into RGB (Red, Green, Blue) space through inverse DCT transform. The LFS feature extracts local frequency domain statistical information of the image and is more sensitive to abnormalities in details. The LFS feature uses sliding window DCT technology to retain the structural information of the image, making it compatible with CNN networks (Convolutional Neural Networks). The model uses sliding window DCT transform on the input image and adaptively collects statistical information of each grid. Finally, the model recombines the frequency statistical information into a multi-channel spatial mapping that shares the same layout as the input image. The local statistical information in each sliding window will be converted into a vector after the above transformation.

[0089] In this embodiment, the FAD feature can adaptively decompose the frequency domain features of the image using a learnable filter, and can more accurately detect forgery traces in different frequency bands.

[0090] In the second embodiment, the step S20 includes:

[0091] Step S201: Fusing the frequency-aware decomposition feature with the local frequency statistical feature to obtain a fused feature.

[0092] Step S202: inputting the face image to be detected and the fusion features into a target bidirectional long short-term memory network model to obtain temporal feature information.

[0093] Step S203: Fusing the temporal feature information with the fusion feature to obtain a confidence level, wherein the confidence level is used to characterize the anti-counterfeiting detection result of the face image to be detected.

[0094] It should be understood that when face image video data enters the model (the overall structure of the model is as follows Figure 3As shown), the present application first extracts two frequency domain features from the original video sequence, namely the frequency-aware decomposition feature FAD feature (Frequency-Aware Decomposition) and the local frequency statistics feature LFS feature (Local Frequency Statistics), and fuses their features in the network through the fusion module Fusion Box. Then, the model passes the original video sequence and the output of the Fusion Box as input into the bidirectional attention LSTM network structure BLA (Bi-LSTM with Attention) that combines the dual dimensions of spatial frequency, thereby extracting richer temporal feature information that integrates spatial frequency information from the video sample set data. Finally, the multi-dimensional temporal features extracted by the bidirectional LSTM are fused with the aforementioned frequency domain features to achieve effective detection of forged face videos (such as AI face-changing, etc.).

[0095] Traditional methods use manually designed fixed filters to extract frequency domain features, which cannot fully cover all image patterns. FAD features use learnable filters to adaptively decompose image frequency domain features and can more accurately detect forgery traces in different frequency bands. The model contains multiple binary filters f base , which divides the frequency of the image into three frequency bands: low, medium, and high. Since fixed filters are difficult to adaptively capture the forgery patterns in the image, the model also contains three learnable filters f w , by combining the above two filters weightedly, the decomposed frequency domain features can be extracted.

[0096] After the image is input, the model first performs a DCT (Discrete Cosine Transform) transform on it, then inputs it into a filter for feature extraction, and finally performs an inverse DCT transform to reconstruct the image.

[0097] The FAD feature extracts frequency domain features and transforms them into RGB space through inverse DCT transform. The LFS feature extracts local frequency domain statistical information of the image and is more sensitive to anomalies in details. The LFS feature uses sliding window DCT technology to retain the structural information of the image, making it compatible with the CNN network. The model uses sliding window DCT transform on the input image and adaptively collects statistical information of each grid. Finally, the model recombines the frequency statistical information into a multi-channel spatial mapping that shares the same layout as the input image. The local statistical information in each sliding window will be converted into a vector after the above transformation.

[0098] After obtaining the FAD features and LFS features of the input image, the model will fuse the features of the two in the network through the Fusion Block. The fusion module further fuses the FAD features and LFS features. First, the FAD features and LFS features are convolved to form an Attention Map. The FAD features and LFS features are multiplied with the Attention Map to obtain F a and L a , FAD characteristics and L a Addition, LFS features and L a Add together to complete feature fusion.

[0099] The Fusion Block input is two layers. After concat (connection), 1x1 convolution is used to reduce the channel, and then N Rep (Re-Parameterization) 3x3 convolution and 3x3 convolution are composed. The two layers are output at the same time, and then concat is used to get the final output. Features in the dual-path grid are fused.

[0100] Extract real-time face video data, detect and locate the face in the face video data, extract the LFS features and FAD features of the face video image data, use the fusion feature module to fuse the dual-path grid features, and the fused features are further extracted and fused by Dense Block, and then passed into the bidirectional long short-term memory neural network (bidirectional LSTM) for inter-frame information extraction. In order to enable the model to better capture key information, this application proposes a bidirectional attention LSTM network structure BLA that combines the dual dimensions of space and frequency. The attention mechanism is added to the bidirectional LSTM, allowing the model to dynamically focus on specific parts of the input sequence and extract the time-varying features of the video sequence from the two dimensions of spatial and frequency domains. This part of the network structure is as follows Figure 4 As shown, represents the attention mechanism, represents the weighted sum. Assume that the attention mechanism is expressed as Attn(·), the bidirectional LSTM is expressed as BiLSTM(·), and A t ,B t are respectively input from the tth frame X t The extracted frequency domain and spatial domain features, concat(·) represents the concatenation operation, Output(·) represents the output of the BLA module, that is, the extracted time series features combined with the spatial frequency features, and the remaining physical quantities are special intermediate state features generated during the model calculation process and have no practical significance. The specific calculation method is as follows:

[0101] f(A t ),f(At )′=BiLSTM(A t )

[0102]

[0103] a(A)=Attn(e(A 1 ),e(A 2 )…,e(A t ))

[0104] s(B t ),s(B t )′=BiLSTM(B t )

[0105]

[0106] a(B)=Attn(e(B 1 ),e(B 2 )…,e(B t ))

[0107]

[0108] BLA can capture the dynamic changes of features in face videos, because videos are a series of continuous frames in which the features of face images change over time. The structure can model these time series data and learn the feature representation of each time step, thereby detecting abnormal temporal patterns.

[0109] BLA combines forward and backward long short-term memory neural networks to capture bidirectional dependencies in video sequence data, which helps the model better understand the contextual relationship of facial features in video sequences and improves sensitivity to possible forgeries in videos. The forward LSTM accepts the input sequence and processes it along the time step. At each time step, it calculates the hidden state and cell state of the current time step based on the current input and the hidden state of the previous time step. The forward LSTM controls the flow of information through a gating mechanism (forget gate, input gate, and output gate), which helps to process long video sequence data and capture long-term dependencies. The backward LSTM is similar to the forward LSTM, but it processes the input sequence in the opposite direction of the time step and is mainly responsible for capturing reverse dependencies in video sequence data.

[0110] BLA obtains the final output by concatenating the hidden states of the forward LSTM and the backward LSTM, so that the output of each time step contains the information of the past and future frames while containing the data features of the current frame itself.

[0111] Typically, a LSTM-based recurrent neural network consists of four parts: xi ,θ hi ,θ ci ,b i The input gate i t , with a weight matrix θ xf ,θ hf ,θ cf ,b f The forget gate t , with a weight matrix θ xc ,θ hc ,θ cc ,b c The state gate C t , with a weight matrix θ xo ,θ ho ,θ co ,b o The output gate O t , all gates pass the current input x t , the hidden state h generated by the previous frame t-1 , and the past state c t-1 , to decide whether to accept the input or forget the previously stored memory, σ represents the sigmoid function σ(x) = (1 + ex) - 1, t still represents the number of frames, and ht represents the hidden state of the current frame. The specific calculation method is as follows:

[0112] f t =σ(θ xf x t +θ hf h t-1 +θ cf c t-1 +b f )

[0113] i t =σ(θ xi x t +θ hi h t-1 +θ ci c t-1 +b i )

[0114] c t =i t g t +f t c t-1

[0115] g t =tanh(θ xc x t +θ hc h t-1+θ cc c t-1 +b c )

[0116] o t =σ(θ xo x t +θ ho h t-1 +θ co c t +b o )

[0117] h t =o t tanh(c t )

[0118] The introduction of the attention mechanism strengthens the model's attention to important information in the sequence data, improving the model's expressiveness and performance. At each time step, BLA takes the hidden states of the forward LSTM and backward LSTM as input, and uses a parameterized function to calculate the time-based attention weight and the space-based attention weight. The calculated attention weights are then used to weight the sum of the hidden states of the forward LSTM and backward LSTM to obtain the importance of the inputs at different time steps and the different positions of the input feature map to the model. The model then uses the weighted sequence representation to focus on important position information such as facial key points and light, and important time information such as the duration of video fluctuations, and further captures and detects the bidirectional relationship and consistency in the time, space, and frequency domains, and finally outputs credibility.

[0119] Reference Figure 5 , Figure 5 This is a flow chart of the third embodiment of the face forgery detection method of the present applicant. Based on the above embodiments, the third embodiment of the face forgery detection method of the present applicant is proposed.

[0120] In the third embodiment, before step S10, the method further includes:

[0121] Step S01: obtaining video frame data to be detected, and performing face image segmentation on the video frame data to obtain a face image to be detected.

[0122] It should be understood that in order to effectively identify identity fraud using static forgery means and improve the security of identity authentication, in this embodiment, before performing anti-counterfeiting detection on the face image to be detected, liveness detection is also performed on the face image to be detected through an instruction liveness detection method and a colorful silent liveness detection method. Only after the liveness detection result is passed, the anti-counterfeiting detection step is continued.

[0123] Furthermore, in order to detect the authenticity of a face, it is first necessary to eliminate the interference of other image information in the video. The step S01 includes: obtaining the video frame data to be detected, and performing face image segmentation on the video frame data through the target detection model to obtain a segmented image; obtaining the eye position in the segmented image, and calculating the face image rotation angle according to the eye position; rotating the segmented image according to the face image rotation angle to obtain a face image to be detected.

[0124] It should be understood that in order to perform the authenticity detection of the face, it is first necessary to eliminate the interference of other image information in the video. Therefore, the process of this embodiment first adopts a target detection model (such as a YOLO (You Only Look Once) detection model, which can be specifically YOLOv5) to identify and segment the face in the video frame, and provide independent face image data for the subsequent detection process. The structure of the YOLOv5 model mainly includes four parts: an input end, a trunk, a neck, and a prediction part. At the input end, data enhancement technology is used to randomly scale, crop, and splice the image to make the training set data distribution more uniform. The trunk and the neck perform feature extraction and fusion operations on the input image. The prediction part performs non-maximum suppression on the target detection frame, thereby obtaining the optimal face detection frame and segmenting it. Through this series of preprocessing steps, independent face image data can be effectively separated from the video, providing basic data support for subsequent face authenticity detection.

[0125] After segmenting the face image, considering the possible tilt problem of the video, the face needs to be aligned. Face alignment means rotating the skewed face image so that the straight line determined by the two eyes of the person in the image remains in a horizontal position. Take the line AB between the two eyes of the face image as the hypotenuse, and then draw the vertical and horizontal right angles respectively, so as to construct the following Figure 6 The right triangle ABC is shown:

[0126] The formula for the cosine value of the rotation angle of a face image is as follows:

[0127]

[0128] Among them, a, b, and c represent the three sides of the triangle, and A represents the angle corresponding to the side a. Then, the face image video data is obtained, and the face image video data is input into the feature extraction model, and feature extraction and reasoning are performed on the face image data to obtain the confidence of the face image.

[0129] Step S02: performing liveness detection on the face image to be detected by using a command liveness detection mode and a colorful silent liveness detection mode to obtain a liveness detection result of the face image to be detected.

[0130] It is understandable that in this application, the liveness detection method mainly includes two methods: command liveness detection and colorful silent liveness detection. Command liveness detection refers to the liveness verification of the user through specific user actions, such as smiling, opening the mouth, shaking the head left and right, nodding, etc., to exclude the possibility of attackers using static photos or masks to deceive. Colorful silent liveness detection, on the other hand, identifies the liveness of the user by extracting the tiny movement features of the user's face in the picture or video. The combination of these two liveness detection methods can effectively identify the use of static forgery means for identity deception, thereby improving the security of identity authentication.

[0131] Furthermore, in order to improve the accuracy of liveness detection, the step S02 includes: identifying different facial movements in the face image to be detected to obtain an instruction liveness detection result; detecting several consecutive frames of the face image to be detected to obtain a colorful silent liveness detection result; and generating a liveness detection result of the face image to be detected based on the instruction liveness detection result and the colorful silent liveness detection result.

[0132] It should be understood that the main technologies of instruction liveness detection are: 1) detecting faces in images; 2) identifying different facial movements; 3) determining whether the user has passed liveness detection.

[0133] The main technologies of colorful silent liveness detection are: 1) Detecting faces in images; 2) Detecting facial images of several consecutive frames to determine whether the user has passed the liveness detection.

[0134] Accordingly, the step S10 includes:

[0135] Step S10 ′: when the liveness detection result is detection passed, extracting frequency-aware decomposition features and local frequency statistical features of the face image to be detected.

[0136] It should be understood that this application proposes a face forgery detection method, which mainly focuses on the following three aspects:

[0137] First, consider the factor of video stability. In real videos, facial information will maintain a relatively stable position and angle, and the relative position of facial information will remain stable even in motion. However, if the face in the video has multiple subtle changes in position, angle or size, it may indicate that a face-swapping operation has occurred. This application uses deep learning related technologies to extract face-related features to address this problem.

[0138] Secondly, consider the consistency of facial key points. Using facial key point information, such as the positions of eyes, nose, mouth, and face edges, in real videos, the positions of these facial key points will remain consistent within a certain range. However, if the positions of facial key points in the video change dramatically between different frames, it may indicate that a face-swapping operation has occurred. This application uses a deep learning method to better extract facial key point consistency features.

[0139] Finally, light consistency is also considered. Light is an important feature that has a significant impact on the realism of a face. If the lighting conditions of a face in a video are significantly inconsistent between different frames, it may also be a sign of face swapping. This application uses a deep learning processing method to better extract the consistency features of ambient light in video data.

[0140] In summary, the face anti-counterfeiting detection solution of the present application includes: first, liveness detection is performed, then face anti-counterfeiting detection is performed on the user, including detection of face masks, face pictures, etc., and finally relevant detection is performed on the AI ​​face-changing situation.

[0141] For ease of understanding, refer to Figure 7 This invention is provided for illustration, but is not intended to limit the present application. Figure 7 This is an overall flow chart of an embodiment of the face forgery detection method of the present application. Figure 7 middle:

[0142] First, obtain the portrait video data, identify and extract the face position information. Then, perform liveness detection to determine whether the portrait is a real person. If it is judged to be non-live (mask or picture), the process is terminated. If the liveness detection passes, the face anti-counterfeiting detection is performed next, including detecting whether it is a mask or picture data. Finally, detect whether the video is an AI face-changing video. If the detection passes, the user can perform subsequent video signing operations. If the detection fails, a prompt will be given and the user will not be allowed to perform video signing related processes.

[0143] Among them, in the liveness detection part, this application proposes a liveness detection method. In the user identity authentication link, colorful liveness and interactive liveness detection methods are used to perform liveness detection on the portrait data in the video to be detected, and identify whether the video portrait data is mask data or portrait picture data.

[0144] In the face anti-counterfeiting detection part, this application proposes a face anti-counterfeiting detection method based on deep learning. The YOLOv5 model structure is used to train on the prepared sample set, adjust the model parameters, and use the face anti-counterfeiting video dataset to train the face anti-counterfeiting detection model.

[0145] In the face anti-counterfeiting detection part, this application proposes a face anti-counterfeiting detection method based on deep learning. The YOLOv5 model structure is used to train on the prepared sample set, adjust the model parameters, and use the face anti-counterfeiting video dataset to train the face anti-counterfeiting detection model.

[0146] Then, the video is preprocessed, and the input video is preprocessed, including noise reduction, color balancing and other operations to improve the quality and clarity of the video, and it is segmented into video frames.

[0147] Secondly, the face is detected and the YOLOv5 model is used to detect and locate the face in the video and extract the face area.

[0148] Next, anti-counterfeiting detection is performed on the face, and the face image video is input into the model for feature extraction and inference to obtain the credibility of the video image data. Combined with the liveness detection method, it is again determined whether it is video data, mask portrait data, or picture portrait data, and whether the face image is alive and whether AI face-changing technology is used based on the credibility threshold.

[0149] Finally, the detection results are output, including whether the person is alive, whether AI face-changing technology is used, and confidence level.

[0150] It should be understood that existing face forgery detection technologies are mainly targeted at specific fields and lack attention to complex scenarios. The 5G e-signature scenario is in a complex detection environment. Attackers may use traditional non-living attack methods or use advanced AI face-changing technology, which brings challenges to face forgery detection.

[0151] This application adopts a face forgery detection method based on deep learning, combined with a liveness detection algorithm, to extract the facial features and time series features of each frame of the video to solve the problem of face anti-counterfeiting detection in complex natural scenes.

[0152] In the 5G e-signature scenario, if the attacker uses AI face-changing attacks, the forged details may be lost due to the influence of network transmission and resolution. Traditional frequency-domain-based detection methods often ignore the correlation between video frames and fail to fully utilize the temporal inconsistency of face forgery videos; while time-domain-based detection methods are easily affected by factors such as noise and compression, and have poor robustness in detecting subtle forgeries. Both methods have their own advantages and disadvantages and need to complement each other.

[0153] Therefore, this application proposes a face forgery detection method that integrates frequency domain, time domain, and spatial domain features, and combines liveness detection and face recognition algorithms to solve various face forgery detection problems in 5G e-signature scenarios and improve the comprehensiveness and robustness of detection.

[0154] Before performing anti-counterfeiting detection on the face image to be detected, this embodiment also performs liveness detection on the face image to be detected through a command liveness detection method and a colorful silent liveness detection method. Only after the liveness detection result is passed, the anti-counterfeiting detection step is continued, thereby effectively identifying the use of static forgery means to commit identity fraud, thereby improving the security of identity authentication.

[0155] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the face forgery detection method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0156] This application also provides a face forgery detection device, please refer to Figure 8 , the face forgery detection device comprises:

[0157] A feature extraction module 10, used to extract frequency-aware decomposition features and local frequency statistical features of the face image to be detected;

[0158] The anti-counterfeiting detection module 20 is used to perform anti-counterfeiting detection on the face image to be detected based on the frequency awareness decomposition features, the local frequency statistical features and the target bidirectional long short-term memory network model to obtain the anti-counterfeiting detection result of the face image to be detected, wherein the target bidirectional long short-term memory network model is a bidirectional attention long short-term memory network model that combines the dual dimensions of space and frequency.

[0159] The face forgery detection device provided by the present application adopts the face forgery detection method in the above embodiment, which can solve the technical problem that the related face forgery attack detection methods are mainly targeted at specific fields and lack attention to complex scenes, resulting in low accuracy. Compared with the prior art, the beneficial effects of the face forgery detection device provided by the present application are the same as the beneficial effects of the face forgery detection method provided by the above embodiment, and other technical features in the face forgery detection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0160] In one embodiment, the anti-counterfeiting detection module 20 is also used to fuse the frequency-aware decomposition feature with the local frequency statistical feature to obtain a fused feature; input the face image to be detected and the fused feature into the target bidirectional long short-term memory network model to obtain time series feature information; fuse the time series feature information with the fused feature to obtain confidence, wherein the confidence is used to characterize the anti-counterfeiting detection result of the face image to be detected.

[0161] In one embodiment, the feature extraction module 10 is also used to perform discrete cosine transform on the face image to be detected, and perform feature extraction on the changed face image through a filter to obtain frequency-aware decomposition features of the face image to be detected, wherein the filter is a weighted combination of multiple binary classification filters and multiple learnable filters, and the multiple binary classification filters divide the frequency of the face image to be detected into multiple frequency bands; extract local frequency domain statistical information of the face image to be detected by sliding window discrete cosine transform, and generate local frequency statistical features of the face image to be detected based on the local frequency domain statistical information.

[0162] In one embodiment, the face forgery detection device further includes:

[0163] A liveness detection module is used to obtain video frame data to be detected, and perform face image segmentation on the video frame data to obtain a face image to be detected; perform liveness detection on the face image to be detected by using a command liveness detection mode and a colorful silent liveness detection mode to obtain a liveness detection result of the face image to be detected;

[0164] Correspondingly, the feature extraction module 10 is also used to extract the frequency-aware decomposition features and local frequency statistical features of the face image to be detected when the liveness detection result is detection passed.

[0165] In one embodiment, the liveness detection module is also used to identify different facial movements in the facial image to be detected to obtain an instruction liveness detection result; detect several consecutive frames of facial images to be detected to obtain a colorful silent liveness detection result; and generate a liveness detection result of the facial image to be detected based on the instruction liveness detection result and the colorful silent liveness detection result.

[0166] In one embodiment, the liveness detection module is also used to obtain video frame data to be detected, and perform face image segmentation on the video frame data through a target detection model to obtain a segmented image; obtain the position of human eyes in the segmented image, and calculate the rotation angle of the face image based on the position of human eyes; rotate the segmented image according to the rotation angle of the face image to obtain the face image to be detected.

[0167] The present application provides a face forgery detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the face forgery detection method in the above-mentioned embodiment one.

[0168] Reference below Fig. 9, which shows a schematic diagram of the structure of a face forgery detection device suitable for implementing the embodiment of the present application. The face forgery detection device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The face forgery detection device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0169] like Fig. 9 As shown, the face forgery detection device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a ROM (Read Only Memory) 1002 or a program loaded from a storage device 1003 to a RAM (Random Access Memory) 1004. In RAM 1004, various programs and data required for the operation of the face forgery detection device are also stored. The processing device 1001, ROM 1002, and RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, an LCD (Liquid Crystal Display), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the face forgery detection device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a face forgery detection device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0170] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0171] The face forgery detection device provided by the present application adopts the face forgery detection method in the above embodiment, which can solve the technical problem that the related face forgery attack detection methods are mainly targeted at specific fields and lack attention to complex scenes, resulting in low accuracy. Compared with the prior art, the beneficial effects of the face forgery detection device provided by the present application are the same as the beneficial effects of the face forgery detection method provided by the above embodiment, and the other technical features of the face forgery detection device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0172] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0173] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0174] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the face forgery detection method in the above-mentioned embodiment.

[0175] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory) or flash memory, optical fiber, CD-ROM (CD-Read Only Memory, portable compact disk read-only memory), optical storage device, magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0176] The computer-readable storage medium may be included in the face forgery detection device; or may exist independently without being assembled into the face forgery detection device.

[0177] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the face forgery detection device, the face forgery detection device executes the face forgery detection method in the above embodiment.

[0178] The computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a LAN (Local Area Network) or a WAN (Wide Area Network), or it can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0179] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0181] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned face forgery detection method, and can solve the technical problem that the related face forgery attack detection methods are mainly targeted at specific fields and lack attention to complex scenes, resulting in low accuracy. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the face forgery detection method provided by the above-mentioned embodiment, and will not be repeated here.

[0182] The present application also provides a computer program product, including a computer program, which implements the above-mentioned face forgery detection method when executed by a processor.

[0183] The computer program product provided by the present application can solve the technical problem that the related face forgery attack detection methods are mainly targeted at specific fields and lack attention to complex scenes, resulting in low accuracy. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the face forgery detection method provided by the above embodiment, and will not be repeated here.

[0184] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for detecting forged faces, characterized in that: The face forgery detection method comprises: Extract frequency-aware decomposition features and local frequency statistical features of the face image to be detected; Based on the frequency awareness decomposition features, the local frequency statistical features and the target bidirectional long short-term memory network model, anti-counterfeiting detection is performed on the face image to be detected to obtain an anti-counterfeiting detection result of the face image to be detected, wherein the target bidirectional long short-term memory network model is a bidirectional attention long short-term memory network model that combines the dual dimensions of space and frequency.

2. The face forgery detection method according to claim 1, characterized in that: The anti-counterfeiting detection is performed on the face image to be detected based on the frequency awareness decomposition feature, the local frequency statistical feature and the target bidirectional long short-term memory network model to obtain the anti-counterfeiting detection result of the face image to be detected, including: Fusing the frequency awareness decomposition feature with the local frequency statistical feature to obtain a fused feature; Inputting the face image to be detected and the fusion features into a target bidirectional long short-term memory network model to obtain temporal feature information; The temporal feature information is fused with the fusion feature to obtain a confidence level, wherein the confidence level is used to characterize an anti-counterfeiting detection result of a face image to be detected.

3. The face forgery detection method according to claim 1, characterized in that: The step of extracting frequency-aware decomposition features and local frequency statistical features of the face image to be detected includes: Performing discrete cosine transformation on the face image to be detected, and extracting features of the changed face image through a filter to obtain frequency-aware decomposition features of the face image to be detected, wherein the filter is a weighted combination of multiple binary classification filters and multiple learnable filters, and the multiple binary classification filters divide the frequency of the face image to be detected into multiple frequency bands; The local frequency domain statistical information of the face image to be detected is extracted by sliding window discrete cosine transform, and the local frequency statistical features of the face image to be detected are generated according to the local frequency domain statistical information.

4. The face forgery detection method according to any one of claims 1 to 3, characterized in that: Before extracting the frequency awareness decomposition features and local frequency statistical features of the face image to be detected, the method further includes: Acquire video frame data to be detected, and perform face image segmentation on the video frame data to obtain a face image to be detected; Performing liveness detection on the face image to be detected by using a command liveness detection mode and a colorful silent liveness detection mode to obtain a liveness detection result of the face image to be detected; Accordingly, the step of extracting the frequency-aware decomposition features and local frequency statistical features of the face image to be detected includes: When the liveness detection result is detection passed, frequency awareness decomposition features and local frequency statistical features of the face image to be detected are extracted.

5. The face forgery detection method according to claim 4, characterized in that: The method of performing liveness detection on the face image to be detected by using the command liveness detection method and the colorful silent liveness detection method to obtain the liveness detection result of the face image to be detected includes: Identifying different facial movements in the face image to be detected to obtain a command liveness detection result; Detect several consecutive frames of face images to be detected to obtain colorful silent liveness detection results; The liveness detection result of the face image to be detected is generated according to the instruction liveness detection result and the colorful silent liveness detection result.

6. The face forgery detection method according to claim 4, characterized in that: The step of acquiring the video frame data to be detected and performing face image segmentation on the video frame data to obtain the face image to be detected includes: Acquire video frame data to be detected, and perform face image segmentation on the video frame data through a target detection model to obtain a segmented image; Obtaining the position of human eyes in the segmented image, and calculating the rotation angle of the face image according to the position of human eyes; The segmented image is rotated according to the rotation angle of the face image to obtain a face image to be detected.

7. A face forgery detection device, characterized in that: The face forgery detection device comprises: A feature extraction module, used to extract frequency-aware decomposition features and local frequency statistical features of the face image to be detected; The anti-counterfeiting detection module is used to perform anti-counterfeiting detection on the face image to be detected based on the frequency awareness decomposition features, the local frequency statistical features and the target bidirectional long short-term memory network model to obtain the anti-counterfeiting detection result of the face image to be detected, wherein the target bidirectional long short-term memory network model is a bidirectional attention long short-term memory network model that combines the dual dimensions of space and frequency.

8. A face forgery detection device, characterized in that: The face forgery detection device includes: a memory, a processor, and a face forgery detection program stored in the memory and executable on the processor. When the face forgery detection program is executed by the processor, the face forgery detection method according to any one of claims 1 to 6 is implemented.

9. A storage medium, characterized in that: The storage medium stores a face forgery detection program, which, when executed by the processor, implements the face forgery detection method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a face forgery detection program, and when the face forgery detection program is executed by a processor, the face forgery detection method according to any one of claims 1 to 6 is implemented.

Images