False face image detection method and system based on image forensics technology

Abstract

The present invention discloses a method and system for detecting false facial images based on image forensics technology. The method mainly includes five steps: Step 1, in the video preprocessing module, Dlib is used to intercept the faces in the original video as a preprocessed data set; Step 2, in the interpolation and underlying feature module, the image is preprocessed by the SRM filter, and then the dimension is reduced by the residual network and the feature extraction is performed by XceptionNet; Step 3, in the background noise module, the PRNU algorithm and the attention mechanism are used to obtain an output vector with background noise information; Step 4, in the video modification detection module, the outputs of the above modules are spliced and passed through the fully connected layer to obtain the modified detection results. Step 5, in the visualization display module, Web interaction technology is used to realize the visualization display of the results. The present invention can effectively improve the accuracy of false face detection, which is of great significance for maintaining a healthy and green network environment.

Description

Technical Field

[0001] The present invention relates to a false face image detection method, and in particular to a false face image detection method and system based on image forensics technology. Background Art

[0002] With the rapid development of the media, the general public often relies on digital media to learn about news and social events, with video being the largest source of information. At the same time, people assume that news reports are honest and trustworthy, which creates an opportunity for criminals to exploit. They manipulate images in videos to misinterpret the original content. Therefore, determining the authenticity of videos is crucial for government public opinion management and event control.

[0003] Early researchers proposed methods for detection based on image dictionaries. Zhang Y et al. modeled this from the perspective of image word vectors. They first used a clustering method to transform the underlying pixels into a codebook containing 512 words. Using the codebook, they then converted 64 selected facial landmarks into vectors forming a matrix, which was then used for classification using an SVM. This approach offers the advantage of natural language processing, but it also complicates image representation. Therefore, researchers revisited the research from the perspectives of image localization and feature selection. Yang X et al. analyzed head pose, first localizing the face using 68 facial landmarks. They then used 21 of these landmarks to represent the central region, and constructed a matrix representing head pose using the position information of these 21 nodes. Finally, an SVM was used for classification and prediction. The major breakthrough of this method is that it directly uses the original image's three RGB components for vector description, eliminating the need to reconstruct a separate set of image feature descriptions. However, significant drawbacks remain. Using a few nodes to replace and modify the image representation in the model can significantly result in a loss of input information, leading to suboptimal judgment results.

[0004] At the same time, methods based on high-level image semantics were gradually discovered. Li et al. discovered that because closed-eye photos are rarely included when learning facial features, the characters in the generated videos rarely blink. By extracting features from the eyes and surrounding pixels and leveraging their temporal characteristics using an LSTM to build a blink analyzer, they were able to identify fake videos based on blink frequency. From this point on, researchers discovered a way to approach high-level semantic features. That is, by observing the generated images, they could specifically detect facial features that differ from normal human behavior. However, the biggest drawback is that counterfeiters can overcome these shortcomings by further learning the blinking feature in the model, rendering this detection method ineffective.

[0005] Building on the work of LiY et al., Matern et al. conducted further analysis and research, finding that in modified images, the iris of the human eye sometimes exhibits color mismatches, and that the human eye's reflection of light can blur and make it impossible to distinguish tooth boundaries. Based on these features, a model was established, and after extracting the corresponding unknown features, logistic regression was used for discrimination. This method, while building on previous methods, discovered more advanced semantic features that can be used to detect fake videos, but the drawback remains the same: the model can be retrained to achieve deception.

[0006] Therefore, it can be seen that methods based on high-level image semantic features, whether through extracting feature points from images or selecting high-level semantic features, have flaws. Although the feature point method is simple, because it only selects individual points to represent the entire image, it may cause loss of the original image semantics and reduce the final detection effect. The method of selecting high-level semantic features, on the other hand, has a highly subjective range of artificial definition, resulting in different observations from different people and providing little help for subsequent research based on this. The biggest flaw is that counterfeiters can overcome this shortcoming by further learning the features currently being detected. In the context of these shortcomings of high-level image semantic detection, detection methods focusing on underlying pixel semantic features have been put on the agenda.

[0007] The researchers first investigated the characteristics of fake videos, focusing on spatiotemporal continuity. Guer, Zhao Lei, and others first proposed a possible solution based on spatial coherence: both DeepFake and Face2Face generate videos frame by frame, and the original real videos fully conform to temporal coherence, which appears to the viewer as continuity in the characters' movements. However, the model does not take this property into account when generating the resulting frames, resulting in the model not learning this characteristic. As a result, the generated faces may produce a phenomenon similar to slight tearing. To address this problem, the paper proposes detecting the presence of non-tearing features between consecutive images for prediction.

[0008] Darius was the first to fully utilize the concept of neural networks to transform this problem into an image classification problem. Building on InceptionNet, the authors redesigned a four-layer network structure by adding a dropout structure, achieving even better results. At the same time, there are also methods for determining whether an image is forged by using the discriminator in a GAN model. The first paper used 11 GANs to generate a model. After pre-training on a large-scale database, the ProGAN model demonstrated good generalization, but still had shortcomings in detecting physical scenes. Li et al. further believed that the biggest difference between real and fake images lies in the fact that the spliced-back image is blurred to enhance credibility during DeepFake fabrication. Therefore, the authors used real images, processed with Gaussian noise, as "fake" images for detection in the model without using DeepFake or Face2Face video frames. They also achieved good results, demonstrating that the use of auxiliary operations in the video forgery detection process, rather than solely analyzing facial images, can also achieve good detection results.

[0009] In 2020, Li et al. used the grayscale image to calculate whether there are spliced continuous linear regions in the image. If these regions can be detected, it can indirectly indicate that the image has been modified.

[0010] In 2021, Zhao et al. applied the multi-attention mechanism to this task, implementing a fake face detection model by building a multi-attention network that combines low-level image semantics with high-level semantics. Chen et al. also achieved excellent results in detecting fake face videos generated by Gan by designing a comprehensive detection model that detects facial reproduction, modification, and replacement.

[0011] For the problem of no public training set and test set, et al.'s research created the publicly available "FaceForensics++" dataset for researchers to use. By simplifying the final fully connected layer of the XceptionNet image classification model, they achieved state-of-the-art results with just three rounds of training, achieving 81% accuracy in low-quality video detection. Furthermore, the shift from training with local feature points to training with the entire face reflects a trend from less to more. The authors attempted to train the model using the entire image, rather than just the face region, in an attempt to increase the data volume. However, the results were actually reduced. The reason is straightforward: other image regions do not affect the final binary classification results, so the additional data represents "noise" for the model learning, which in turn reduces the model's effectiveness.

[0012] Although the task of false face recognition was only proposed at the end of 2017, there have been many methods for detection from various aspects in just two or three years. The mainstream method is to regard this problem as a binary image classification problem, which is further divided into two categories: methods based on high-level semantic features and methods based on low-level semantics.

[0013] From a development perspective, low-level semantics-based methods and high-level semantic feature methods have developed in parallel. However, low-level semantics methods have proven superior in terms of results. This is because low-level semantics methods incorporate more information (image range) into the model training than high-level feature methods. This allows the model to learn more details about distinguishing real from fake images, rather than manually selecting features. Furthermore, because high-level semantic methods target overly obvious features, counterfeiters can deceive the corresponding detection methods by improving these targeted features.

[0014] The detection results show that higher video clarity leads to better results. Currently, the accuracy of low-resolution detection is 81%, which still leaves room for improvement. In terms of data, early researchers constructed their own training and test sets before testing their models. This approach, however, hindered direct comparison between models.

[0015] On the other hand, current detection methods all detect the image part generated by the model and ignore smoothing processing. Smoothing processing is an operation that processes the edges of the spliced area. Therefore, if traces of smoothing processing are detected, it can also indirectly confirm that the image layer has been modified. How to detect smoothing processing can also be used as one of the new detection methods.

[0016] Based on the above research background, the present invention aims at the problem of false face video detection by transforming the original classification problem into a positioning problem, using background noise information for detection, and adding detection smoothing processing to determine whether the video has been modified. A false face image detection method and system based on image forensics technology are proposed. Summary of the Invention

[0017] Purpose of the invention: To address the deficiencies of the prior art, the present invention provides a method and system for detecting false facial images based on image forensics technology.

[0018] Technical solution: To achieve the above-mentioned purpose, the present invention adopts the following technical solution:

[0019] A method for detecting fake face images based on image forensics technology, the method comprising the following steps:

[0020] (1) Video preprocessing

[0021] The video is preprocessed to a uniform frame rate. The location information of the image area where the face is located in the video is extracted using Dlib. Based on the location information of the face area, the face feature image of the face in the original video is captured as the preprocessed data set.

[0022] (2) Interpolation and underlying feature detection

[0023] The facial feature image obtained in step (1) is preprocessed by the SRM filter to obtain a new matrix rich in interpolation detection information. The new matrix of interpolation detection information is then fused on the basis of the original facial features and placed in the residual network for dimensionality reduction to obtain fused features. Finally, feature extraction is performed through XceptionNet to obtain an output vector that fuses the underlying features and interpolation features.

[0024] (3) Background noise detection

[0025] The facial feature image obtained in step (1) is subjected to the PRNU algorithm to obtain a matrix representation with background noise characteristics. Each pixel in the matrix with background noise characteristics represents the feature intensity of whether it has been modified. Then, the self-attention mechanism is used to eliminate the influence of the fixed threshold and obtain an output vector with background noise information;

[0026] (4) Video modification detection

[0027] The output vector of the fused underlying features and interpolated features obtained in steps (2) and (3) is concatenated with the output vector with background noise information in terms of feature dimension, and then passes through a fully connected layer to obtain the detection result of video modification;

[0028] (5) Visualization of results

[0029] Based on the detection result obtained in step (4), combined with the face position information obtained in step (1), the detection result information is displayed on the Web platform.

[0030] Furthermore, the pre-processing of the video to unify the frame rate in step (1) adopts a detection interval of once every 6 frames;

[0031] In step (1), Dlib is used to extract the location information of the image area where the face is located in the video. Specifically, the open source tool Dlib is used for face detection. Dlib first uses a trained model to detect 68 key points on the face and then returns the location information of the image area where the face is located;

[0032] In step (1), the face in the original video is captured as a preprocessed data set based on the location information of the face area. Specifically, the face of the person is fixed in the center of the image by capturing the face so as to associate the face area with the center area of the image.

[0033] Furthermore, step (2) specifically includes the steps of:

[0034] (21) According to step (1), a data set P containing facial feature images is obtained, with dimensions of (N, H, W, C), where H and W are the height and width of the image, C is the number of color channels, and N is the number of images. The face area in the image is separately intercepted and recorded as P′; after processing, P is interpolated and detected through a specific SRM filter to obtain a new matrix rich in interpolation detection information; the specific SRM filter is composed of a residual matrix, and the coefficient of each element in the residual matrix is determined by the distance between the pixel point and the center node and the current order: the coefficient Where t represents the minimum number of jumps required for the current pixel to reach the primary pixel, and n represents the order of the current analysis.

[0035] (22) fusing the new matrix rich in interpolation detection information obtained in step (21) with the original facial feature information, and then placing it in a residual network for dimensionality reduction to obtain fused features, wherein the residual network structure is composed of a stack of convolutional layers, pooling layers, and fully connected layers;

[0036] (23) The fusion features obtained in step (22) are further extracted through XceptionNet, and finally the features are spliced to obtain the output vector S′ of the fusion underlying features and interpolation features o .

[0037] Furthermore, step (3) specifically includes:

[0038] The dataset P containing facial feature images obtained in step (1) is subjected to the PRNU algorithm to obtain a matrix representation with background noise characteristics. The calculation process of the PRNU algorithm is shown in Formula 1:

[0039]

[0040]

[0041] Where: Y represents the final image output, which represents the grayscale image in this invention, I is the input signal, k is the PRNU parameter, μ is the additional noise combination of multiple interference sources, g is the color channel gain, γ is the correction factor, which is generally 0.45, θ is the quantization noise, I 0 is the signal output when there is no noise, θ 0 is a collection of mutually independent random noises;

[0042] Then, the residual noise W is calculated by comparing it with the original image output Y as shown in Formula 2:

[0043] W=YI 0′=Yk+I 0 -I 0′ +(I 0 -Y)k+θ 0 =Yk+θ 0′ (2)

[0044] Where: θ 0′ represents θ 0 , I 0 -I 0′ and(I 0 -I)kThe sum of these three items, for all images, for the sake of simplicity, θ 0′ It is generally set to obey the variance σ 2 , a Gaussian distribution with mean 0;

[0045] Finally, by estimating the log likelihood of k in Formula 2, the global average parameter of the PRNU algorithm is obtained. As shown in Formula 3,

[0046]

[0047] The subscript i represents dividing the original image into i parts, and calculating each part one by one, and finally obtaining is a matrix of the original image size;

[0048] First calculate the average noise parameter of the entire image Then, by segmenting the global image, the same principle is used to calculate the local k i , and then alone with By comparing the threshold, we can conclude whether each part has been modified;

[0049] K i and The ratio is mapped to the neurons, removing the influence of the fixed threshold on the model, and obtaining the output matrix set P i . ;

[0050] Use the Attention mechanism to eliminate the impact of fixed thresholds:

[0051] The input of the Attention mechanism consists of three parts: query (Q), key (K), and value (V). K and V have a one-to-one correspondence, and V is completely consistent with K. By calculating the correlation between Q and K and taking it as the weight of V, the weighted sum of V is taken to obtain the output result, as shown in Formula 4:

[0052]

[0053] Where d represents the dimension of the elements in Query and Key, which plays the role of scaling the dot product operation result.

[0054] A visual display system for a false face image detection method based on image forensics technology, comprising: a user interaction module, a background execution module, and the background execution module can be further divided into a data management module, a preprocessing module, and an analysis module;

[0055] The user interaction module is used to upload user pictures and video data and visualize the analysis results;

[0056] The pre-processing module is used to locate and segment faces in pictures or videos in the original material library for subsequent analysis module to analyze;

[0057] The analysis module is used to determine whether the face is forged based on the image data input by the pre-processing module, and finally save the analyzed image into the processed material library and generate the analysis results;

[0058] The data management module is used to store and manage the data in each module and the output data. According to the interaction and logical order of the functions of each module, the data management module is divided into an analysis task index library, an original material library, a processed material library and an analysis result library.

[0059] Beneficial effects: Compared with the prior art, the present invention adopts the above technical solution and has the following advantages:

[0060] (1) We approach the problem from two perspectives: background noise and interpolation detection. Based on the original underlying semantic detection, we combine the results of background noise and interpolation detection to establish a multi-strategy detection model and propose a detection algorithm, thus achieving the goal of false face detection.

[0061] (2) Using the self-attention mechanism to fuse background noise information, a multi-scale background noise feature detection method was designed to quantitatively analyze the feature differences between the face region and the entire image, making the detection results more accurate;

[0062] (3) Starting from the interpolation operation in the counterfeiting process, we consider detecting the interpolation operation in the process for the first time, and assist in video judgment from the side. We design an interpolation and underlying feature detection method based on rich steganalysis (SRM), and fuse it with multi-scale background noise information to determine whether the image in the video has been modified, making the model more robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is the second-order schematic diagram of SRM;

[0064] Figure 2 It is a structural diagram of the convolutional residual network;

[0065] Figure 3 It is the structure diagram of the interpolation and underlying feature detection module;

[0066] Figure 4 This is a schematic diagram of the use of the PRNU algorithm;

[0067] Figure 5 It is a schematic diagram of multi-scale background noise extraction;

[0068] Figure 6 This is the structure diagram of the multi-scale background noise extraction method based on the self-attention mechanism;

[0069] Figure 7 It is the overall framework diagram of the present invention;

[0070] Figure 8 This is the interface diagram for user uploading data;

[0071] Figure 9 This is a visualization interface diagram of the dataset experimental results;

[0072] Figure 10 This is the interface diagram for returning test results. DETAILED DESCRIPTION

[0073] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0074] The following is only one embodiment of the present invention. The present invention has many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications based on the present invention. These corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

[0075] like Figure 7 As shown, the false face image detection method based on image forensics technology of the present invention includes the following steps:

[0076] 1. Video preprocessing

[0077] The preprocessing of the video to unify the frame rate is to address the problem of too small differences between adjacent video frames. The detection interval is set to once every 6 frames to avoid excessive repetition of the same data set. This method uses Dlib to extract the position information of the image area where the face is located in the video. Specifically, the open source tool Dlib is used for face detection. Dlib first uses the trained model to detect 68 key points (Land Mark) on the face and then returns the position information of the image area where the face is located. Then, according to the position information of the face area, the face in the original video is cut off as the preprocessed data set. Specifically, the face of the person is fixed in the center of the image by cutting off, so that the face area can be associated with the center area of the image.

[0078] 2. Interpolation and underlying feature detection

[0079] (1) According to the previous step, a picture set P containing the face and its surrounding area is obtained, with dimensions of (N, H, W, C), where H and W are the height and width of the image, and C is the number of color channels. The face area in the picture is cut out separately and recorded as P'. After processing, P is interpolated and detected through the SRM filter to obtain a new matrix rich in interpolation detection information. SRM is composed of a residual matrix, as shown in Table 1. Among them, the order represents the number of pixels that should be calculated in addition to the current analysis pixel, that is, the number of pixels across = order + 1. The table only lists the first six horizontal orders and the default main analysis point is close to the left boundary. If other orders are needed, they can be determined by the user. The coefficient of each element in the residual matrix is determined by the distance between the pixel point and the center node and the current order: coefficient Where t represents the minimum number of jumps required for the current pixel position to reach the primary pixel position, and n represents the order of the current analysis.

[0080] Table 1 SRM residual matrix

[0081]

[0082] In actual use, its form and method are very similar to today's convolution kernel, and its form is as follows Figure 1 shown.

[0083] Figure 1 Taking the second-order SRM as an example, the first row lists the convolution kernels in all four directions: horizontal, vertical, oblique, and reverse oblique. It assumes that the convolution results of the corresponding kernels with the input image matrix are A, B, C, and D, respectively. The maximum and minimum values of the different direction combinations are used as the input to the next layer. The second and third rows show four possible combinations and their corresponding outputs: 1) Considering only the horizontal direction; 2) Considering both the horizontal and vertical directions; 3) Considering both the horizontal and reverse oblique directions; and 4) Considering all four directions.

[0084] (2) The new matrix rich in interpolation detection information obtained in the previous step is fused with the original facial feature information and then put into the residual network for dimensionality reduction. The residual network structure is as follows: Figure 2 shown.

[0085] (3) The fusion features obtained in the previous step are further extracted through XceptionNet, and finally the features are spliced to obtain the output vector S′ of the fusion underlying features and interpolation features o The overall structure of the interpolation and underlying feature detection module is as follows: Figure 3 .

[0086] 3. Background noise detection

[0087] The facial feature image obtained in video preprocessing is passed through the PRNU algorithm to obtain a matrix representation with background noise characteristics. The calculation process of PRNU is shown in Formula 1. In Formula 1, Y represents the final image output, which represents the grayscale image in this study, I is the input signal, k is the PRNU parameter, μ is the additional noise combination of multiple interference sources, g is the color channel gain, γ is the correction factor, which is generally 0.45, and θ is the quantization noise. 0 is the signal output when there is no noise, θ 0 is a collection of mutually independent random noises.

[0088]

[0089] Then the residual noise W is calculated by comparing it with the original image output Y as shown in Formula 2. 0′ represents θ 0 , I 0 -I 0′ and(I 0 -I)k is the sum of these three items. For all images, in order to simplify the calculation, θ 0′ It is generally set to obey the variance σ 2 , a Gaussian distribution with mean 0.

[0090] W=YI 0′ =Yk+I 0 -I 0′ +(I 0 -Y)k+θ 0 =Yk+θ 0′ (2)

[0091] Finally, the log likelihood of k is estimated by using formula 2. The global PRNU average parameter can be obtained. As shown in formula 3, the subscript i represents dividing the original image into i parts, and calculating each part one by one. A matrix of the original image size.

[0092]

[0093] After preprocessing, the central area of the image can represent the face area. Figure 4 As shown, the left half of the image represents the input grayscale image containing the potential modification area, which represents the input image in the experiment. The shading in it represents the background noise parameter. First, the average noise parameter of the whole image is calculated. Then, by segmenting the global image, the same principle is used to calculate the local k i , and then alone with By comparing the threshold, we can conclude whether each part has been modified.

[0094] In this method, the ratio of the two is used to map to neurons, which can remove the influence of the fixed threshold on the model, so that the algorithm can be integrated into the neural network learning, and finally the output matrix set P can be obtained. i At the same time, since the previous preprocessing process has bound the face area and the image center area, we only need to select the k5 area in the figure for detection.

[0095] In order to solve the problem of how to choose the size of each area, this paper designs a multi-scale background noise feature extraction method suitable for this study, such as Figure 5 shown.

[0096] Figure 5 The three different scale extraction methods among the multi-scale background noise extraction features are shown in Figure 2. The specific algorithms are shown in Table 2.

[0097] Table 2 Multi-scale background noise extraction algorithm

[0098]

[0099]

[0100] In order to obtain an output vector with background noise information, this method uses the Attention mechanism to eliminate the influence of the fixed threshold.

[0101] The input to the Attention mechanism consists of three main components: the query (Q), the key (K), and the value (V). K and V have a one-to-one correspondence, and V is typically identical to K. The output is obtained by calculating the correlation between Q and K, using this as the weight for V, and performing a weighted summation of V. The formula is shown in Figure 4. Here, d represents the dimension of the elements in the query and key, which serves to scale the dot product result. When Q and K are identical, the Attention mechanism, also known as the self-attention mechanism, can capture internal dependencies, offering significant advantages over traditional RNNs or LSTMs.

[0102]

[0103] The Multi-Head Attention mechanism transforms the input text or image multiple times to obtain different Q and V, performs attention calculations, and finally simply concatenates the results.

[0104] The multi-scale background noise detection method of this method first extracts a set of training images from the data set with dimensions (N, H, W, C), where H and W are the height and width of the image, and C is the number of color channels. These samples are then sent to the multi-scale background noise extraction algorithm based on the PRNU algorithm in step (3) to obtain background noise features at multiple different scales. These noise results at different scales are then placed in the Attention module in sequence according to the image segmentation size, allowing the model to automatically extract features to enhance the problem of insufficient robustness of a single scale.

[0105] 4. Video modification detection

[0106] The interpolation obtained by the above method is concatenated with the underlying feature information and background noise information in the feature dimension, and then passed through the fully connected layer to obtain the detection result of video modification.

[0107] 5. Visual display

[0108] This part consists of a user interaction module and a background execution module. The background execution module can be divided into a data management module, a preprocessing module and an analysis module.

[0109] The functions of each module are described as follows:

[0110] (1) User interaction module: mainly responsible for uploading user pictures and video data, and visually displaying the analysis results. The effect of this module is shown as follows Figure 8 .

[0111] (2) Preprocessing module: The task of the preprocessing module is to locate and segment faces in the images or videos in the original material library for subsequent analysis modules to analyze.

[0112] (3) Analysis module: The analysis module is based on the false face detection model of this method. It determines whether the face is a fake face according to the image data input by the pre-processing module, and finally saves the analyzed image into the processed material library and generates the analysis results. The effect of this module is shown as follows: Figure 9 .

[0113] (4) Data management module. The data management module is mainly responsible for storing and managing the data in each module and the output data. It is the basis and important data support for the entire prototype. According to the interaction and logical order of each module function, the data management module is mainly divided into analysis task index library, original material library, processed material library and analysis result library. The effect of this module is shown as follows: Figure 10 .

Claims

1. A false face image detection method based on image forensics technology, characterized in that: The method comprises the following steps: (1) Video preprocessing The video is preprocessed to a uniform frame rate. The location information of the image area where the face is located in the video is extracted using Dlib. Based on the location information of the face area, the face feature image of the face in the original video is captured as the preprocessed data set. (2) Interpolation and underlying feature detection The facial feature image obtained in step (1) is preprocessed by the SRM filter to obtain a new matrix rich in interpolation detection information. The new matrix of interpolation detection information is then fused on the basis of the original facial features and placed in the residual network for dimensionality reduction to obtain fused features. Finally, feature extraction is performed through XceptionNet to obtain an output vector that fuses the underlying features and interpolation features. (3) Background noise detection The facial feature image obtained in step (1) is subjected to the PRNU algorithm to obtain a matrix representation with background noise characteristics. Each pixel in the matrix with background noise characteristics represents the feature intensity of whether it has been modified. Then, the self-attention mechanism is used to eliminate the influence of the fixed threshold and obtain an output vector with background noise information; First, a set of training images are extracted from the dataset with dimensions (N, H, W, C), where H and W are the height and width of the image, and C is the number of color channels. These samples are then fed into the multi-scale background noise extraction algorithm based on the PRNU algorithm in step (3) to obtain background noise features at various scales. These noise results at various scales are then sequentially placed into the Attention module according to the image segmentation size, allowing the model to automatically extract features. (4) Video modification detection The output vector of the fused underlying features and interpolated features obtained in steps (2) and (3) is concatenated with the output vector with background noise information in terms of feature dimension, and then passes through the fully connected layer to obtain the detection result of the video modification; (5) Visualization of results Based on the detection result obtained in step (4), combined with the face position information obtained in step (1), the detection result information is displayed on the Web platform.

2. The false face image detection method based on image forensics technology according to claim 1 is characterized in that: The pre-processing of the video to unify the frame rate described in step (1) adopts a detection interval of once every 6 frames; In step (1), Dlib is used to extract the location information of the image area where the face is located in the video. Specifically, the open source tool Dlib is used for face detection. Dlib first uses the trained model to detect 68 key points on the face and then returns the location information of the image area where the face is located; In step (1), the face in the original video is captured as a pre-processed data set based on the location information of the face area. Specifically, the face of the person is fixed in the center of the image by capturing the face so as to associate the face area with the center area of the image.

3. The false face image detection method based on image forensics technology according to claim 1, characterized in that: Step (2) specifically includes the following steps: (21) According to step (1), a dataset P containing facial feature images is obtained, with dimensions of (N, H, W, C), where H and W are the height and width of the image, C is the number of color channels, and N is the number of images. The face area in the image is individually intercepted and recorded as P'; after processing, P is interpolated and detected through a specific SRM filter to obtain a new matrix rich in interpolation detection information; the specific SRM filter is composed of a residual matrix, and the coefficient of each element in the residual matrix is determined by the distance between the pixel point and the center node and the current order: coefficient , where t represents the minimum number of jumps required for the current pixel to reach the primary pixel, and n represents the order of the current analysis; (22) Fusing the new matrix rich in interpolation detection information obtained in step (21) with the original facial feature information, and then placing it in a residual network for dimensionality reduction to obtain fused features. The residual network structure is composed of a stack of convolutional layers, pooling layers, and fully connected layers; (23) The fusion features obtained in step (22) are further extracted through XceptionNet, and finally the features are spliced to obtain the output vector of the fusion underlying features and interpolation features. .

4. The false face image detection method based on image forensics technology according to claim 1, characterized in that: Step (3) specifically includes: The dataset P containing facial feature images obtained in step (1) is subjected to the PRNU algorithm to obtain a matrix representation with background noise characteristics. The calculation process of the PRNU algorithm is shown in Formula 1: ； ； ； (1) Where: Represents the final image output, represents the grayscale image, is the input signal, is the PRNU parameter, is the combination of additional noise from multiple interference sources, is the color channel gain, is the correction factor, which is 0.45, is the quantization noise, Is the signal output when there is no noise, is a collection of mutually independent random noises; Then output the image with the original image Calculate the residual noise As shown in Formula 2: (2) Where: Represents 、 and The sum of these three items, for all images, for simplicity of calculation, Set to obey the variance , a Gaussian distribution with mean 0; Finally, by estimating the log likelihood of k in Formula 2, the global average parameter of the PRNU algorithm is obtained. As shown in Formula 3, (3) The subscript Represents the original image divided into Each part is calculated one by one, and the final result is is a matrix of the original image size; First calculate the average noise parameter of the entire image , and then by segmenting the global image, the same principle is used to calculate the local , and then alone with By comparing the threshold, we can conclude whether each part has been modified; Will and The ratio is mapped to the neuron, removing the influence of the fixed threshold on the model and obtaining the output matrix set ; Use the Attention mechanism to eliminate the impact of fixed thresholds: The input of the Attention mechanism consists of three parts: query Q, key K, and value V. K and V have a one-to-one correspondence, and V is completely consistent with K. The correlation between Q and K is calculated and used as the weight of V. The weighted sum of V is then taken to obtain the output result, as shown in Formula 4: （4） Where d represents the dimension of the elements in Query and Key, which plays the role of scaling the dot product operation result.

5. A visual display system for the false face image detection method based on image forensics technology according to any one of claims 1 to 4, characterized in that: The system includes: a user interaction module, a background execution module, and the background execution module can be divided into a data management module, a pre-processing module and an analysis module; The user interaction module is used to upload user pictures and video data and visualize the analysis results; The pre-processing module is used to locate and segment faces in images or videos in the original material library for subsequent analysis by the analysis module; The analysis module is used to determine whether the face is forged based on the image data input by the pre-processing module, and finally save the analyzed image into the processed material library and generate the analysis results; The data management module is used to store and manage the data in each module and the output data. According to the interaction and logical order of the functions of each module, the data management module is divided into an analysis task index library, an original material library, a processed material library and an analysis result library.

Images