Explanation and analysis method for counterfeited face

By combining image compression differences, pixel associations, face offsets and depth algorithm feature vectors, the problem of difficult to provide detailed explanations and evidence for fake face recognition in the prior art is solved, and the forged face is realized. Effective identification and interpretation of images improves the credibility and transparency of false identification.

CN119091477APending Publication Date: 2024-12-06HANGZHOU ZHONGKE RUIJIAN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410433849.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-04-11
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing forged face verification technology mainly relies on deep model classification detection, which is difficult to provide detailed explanations and evidence, and may be misjudged or affected by data deviations. It lacks in-depth analysis of subtle traces of images, lacks transparency and traceability, which affects the results. credibility and impartiality.

Method used

Multi-level interpretation analysis method is adopted, including the combination of image compression differences, pixel association and distribution inconsistency, face offset, and depth algorithm feature vectors through Laplace operator, unet++ network, mask2former network and convolutional neural network Perform layer-by-layer interpretation analysis to obtain detailed explanation results of fake face images.

Benefits of technology

It realizes effective identification of four types of forged forms (face change, reenactment, editing, pure synthesis) of forged face images, improves the credibility and transparency of false accusation, provides detailed explanations and verifiable evidence to ensure that credibility and impartiality of the results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091477A_ABST
    Figure CN119091477A_ABST
Patent Text Reader

Abstract

The invention relates to a counterfeited face interpretation and analysis method. The method is suitable for the counterfeited face interpretation and analysis field. According to the technical scheme, the method for interpreting and analyzing the forged face is characterized by comprising the following steps: acquiring a face image to be interpreted and analyzed; performing interpretation analysis on the face image based on the image compression difference to obtain an interpretation analysis result I; performing interpretation analysis on the face image based on pixel association and distribution inconsistency to obtain an interpretation analysis result II; explaining and analyzing the face image based on the face maladjustment to obtain an explaining and analyzing result III; explaining and analyzing the face image based on the depth algorithm feature vector to obtain an explaining and analyzing result IV; and based on the interpretation and analysis results I, II, III and IV, obtaining a counterfeited face interpretation and analysis result. According to the method, the counterfeit images are interpreted and analyzed layer by layer in a cascade mode by combining flaw representation forms possibly existing in the four types of counterfeit images, namely face changing, face replaying, face editing and pure synthesis.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a forged face interpretation and analysis method, which is applicable to the forged face interpretation and analysis field. Background Art

[0002] With the continuous development of deep learning technology, face forgery technology represented by Generative Adversarial Networks (GAN) and diffusion models has become increasingly powerful. They have tampered with or generated a large amount of image data that is indistinguishable from the real thing. While enriching people's lives, these images have also brought new challenges and problems to social stability and security. Through the fermentation of media such as online media, it can have a huge social impact in a very short time. Through the fermentation of media such as online media, it can have a huge social impact in a very short time. Therefore, it is crucial to identify the forgery of important images in order to maintain the credibility of information, social security and fairness.

[0003] At present, the technology for detecting fake faces mainly focuses on the training of classification models with large amounts of labeled data, while purely relying on deep models for image detection only lacks a detailed explanation of the reasons behind the image, making it difficult to provide convincing evidence. Deep learning models may misjudge in some cases or be limited by data bias, and cannot deeply analyze subtle traces in the image or how the traces are generated. To improve the credibility of detection, it is necessary to combine knowledge from multiple fields such as image science, digital forensics, and image processing to explain in a more comprehensive and detailed way why the image is identified as a fake. In addition, transparency and traceability are also key, so that the process and basis of detection can be understood and verified by both professionals and the public, thereby ensuring the credibility and fairness of the results. Summary of the invention

[0004] The technical problem to be solved by the present invention is: to provide a method for explaining and analyzing forged faces in view of the above-mentioned problems.

[0005] The technical solution adopted by the present invention is: a method for explaining and analyzing forged faces, characterized in that it includes:

[0006] Obtaining a facial image to be interpreted and analyzed;

[0007] Based on the difference of image compression, the facial image is interpreted and analyzed, and the interpretation and analysis result I is obtained;

[0008] Based on pixel association and distribution inconsistency, the face image is interpreted and analyzed, and the interpretation and analysis results II are obtained;

[0009] Based on face disorder, the face image is interpreted and analyzed to obtain interpretation and analysis results III;

[0010] Based on the deep algorithm feature vector, the face image is interpreted and analyzed to obtain the interpretation and analysis result IV;

[0011] Based on the explanation and analysis results Ⅰ, Ⅱ, Ⅲ, and Ⅳ, the explanation and analysis results of the forged face are obtained.

[0012] The facial image is interpreted and analyzed based on the image compression difference to obtain the interpretation and analysis result I, including:

[0013] The Laplace operator is applied to the face image to extract the image gradient, and the morphological correction is performed to obtain the interpretation and analysis results.

[0014] The facial image is interpreted and analyzed based on pixel association and distribution inconsistency to obtain interpretation and analysis result II, including:

[0015] The face image is input into the trained unet++ network to obtain the foreign intrusion area with inconsistent pixel association and distribution in the face image.

[0016] The facial image is interpreted and analyzed based on facial disorder to obtain interpretation and analysis result III, including:

[0017] The face image is input into the trained mask2former network to obtain the defective area that is inconsistent with the normal face.

[0018] The facial image is interpreted and analyzed based on the deep algorithm feature vector to obtain the interpretation and analysis result IV, including:

[0019] Input the face image into the convolutional neural network to obtain the high-dimensional features of the face and sparse the features into multi-dimensional vectors;

[0020] When performing a visual display, vectors that tend to be real features are marked in blue, and vectors that tend to be fake features are marked in red. Then the values ​​are sorted and the final feature vector statistics graph is output.

[0021] A forged face interpretation and analysis device, characterized by comprising:

[0022] An image acquisition module, used to acquire a face image to be interpreted and analyzed;

[0023] The explanation and analysis module I is used to explain and analyze the face image based on the image compression difference to obtain the explanation and analysis result I;

[0024] Explanation and analysis module II, used to interpret and analyze the face image based on pixel association and distribution inconsistency, and obtain explanation and analysis result II;

[0025] The explanation and analysis module III is used to explain and analyze the face image based on the face disorder to obtain the explanation and analysis result III;

[0026] The interpretation and analysis module IV is used to interpret and analyze the face image based on the deep algorithm feature vector to obtain the interpretation and analysis result IV;

[0027] The comprehensive analysis module is used to obtain the forged face interpretation analysis results based on the interpretation analysis results I, II, III, and IV.

[0028] A storage medium stores a computer program that can be executed by a processor, characterized in that when the computer program is executed, the steps of the forged face interpretation and analysis method are implemented.

[0029] A forged face interpretation and analysis device comprises a memory and a processor, wherein the memory stores a computer program executable by the processor, and is characterized in that when the computer program is executed, the steps of the forged face interpretation and analysis method are implemented.

[0030] The beneficial effects of the present invention are as follows: forged images can be roughly divided into four categories at the level of forgery purpose: face-changing, face reenactment, face editing, and pure synthesis. The present invention combines the possible defect manifestations of the above four types of forged images and applies a cascaded approach to interpret and analyze the forged images layer by layer. The present invention searches for inconsistent areas in the image based on compression differences. When there is a significant difference in the gradient information between the face and the peripheral area of ​​the face, it indicates that the original image is a forged image with a face-changing face. A deep model is used to find areas with inconsistent pixel distribution and represent them with a binary mask. A model based on an attention mechanism is used to find the parts of the face that are out of alignment and locate foreign intrusion pixels. Feature vectors are extracted from the image, high-dimensional features are visualized, and visual vector features are used to quantify forged features and real features. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of four types of forged face images.

[0032] Figure 2 This is a flowchart for explaining and analyzing forged faces in the embodiment.

[0033] Figure 3 The flowchart of the interpretation and analysis based on image compression difference in the embodiment is shown in FIG.

[0034] Figure 4 This is a diagram showing the difference between the forged and real faces in the embodiment.

[0035] Figure 5 Flow chart of the analysis of pixel association and distribution inconsistency in the embodiment.

[0036] Figure 6 It is a flowchart of interpretation and analysis based on facial disorders in an embodiment.

[0037] Figure 7 The explanation and analysis process of the deep algorithm feature vector in the embodiment DETAILED DESCRIPTION

[0038] Example 1: Face forgery images can be roughly divided into four categories based on the purpose of forgery: face replacement, face reenactment, face editing, and pure synthesis. Figure 1 .

[0039] Face swapping refers to replacing the target face with the source face through migration or encoding and decoding. When explaining and analyzing this type of problem, it is necessary to pay attention to possible defects or inconsistencies between the face swap area and the source area, face edges, etc.

[0040] Face reenactment is also called expression driving, which means making the source face show the target expression, while face editing refers to changing the attribute characteristics of one or several organs of the face. When explaining and analyzing these two types of problems, we need to focus on the facial features and other places that are easy to change.

[0041] Pure synthesis means that the entire image is generated. The forged result comes from the natural image but the natural image part does not exist. When explaining and analyzing this problem, it is necessary to pay attention to the inconsistent parts of the face image.

[0042] like Figure 2 As shown, this embodiment is a forged face interpretation and analysis method, which specifically includes the following steps:

[0043] S1. Obtain a face image to be analyzed.

[0044] S2. Perform interpretation and analysis on the face image based on the image compression difference to obtain interpretation and analysis result I.

[0045] The main process of interpretation and analysis based on image compression differences in this example is as follows Figure 3 As shown in the figure, the input image is applied with the Laplacian operator to extract the image gradient, and then the gradient map is filtered by empirical values ​​such as mean and variance, followed by morphological correction to obtain the final interpretation and analysis result. If a binary mask is extracted, the morphologically corrected image needs to be post-processed.

[0046] After the analysis of the image compression differences, the difference between the results of the forged face image and the normal image is as follows: Figure 4 As shown, the left 1 is the explanation and analysis result of the fake face, and the middle and right 1 are the explanation and analysis results of the real face.

[0047] Forged face images usually involve operations such as editing and synthesis of images. In order to ensure the area adaptability of the synthesized or edited area with the source area, the foreign area is generally expanded or compressed, which leads to the objective existence of two areas with different compression ratios in the forged image. Gradient operators such as Laplace are used to highlight the edge and texture information in the image, so that the distribution of pixel values ​​is changed and the edge strength is enhanced. Through filtering operations such as mean and variance, the potential features of the synthesized or edited part can be extracted. In order to improve the visualization effect of the potential feature part, morphological methods are finally applied for correction. After multiple verifications of a large number of forged images, for images without excessive post-processing operations, the difference between the original image that has not been edited or synthesized and the forged image is relatively large, which shows that the analysis method is reliable and effective.

[0048] S2. Perform interpretation analysis on the face image based on pixel association and distribution inconsistency to obtain interpretation analysis result II.

[0049] The main process of interpretation and analysis based on pixel association and distribution inconsistency is as follows: Figure 5 As shown, the face image (or the mask added in the previous step) is input into the trained unet++ network to obtain the foreign intrusion area with pixel association and distribution inconsistency. The output result is post-processed to obtain a binary mask map.

[0050] In this embodiment, the unet++ network needs to be trained with specific data, and marking the training data only needs to mark areas that are identifiable to the naked eye and have large differences in pixel distribution and association with the original image.

[0051] According to the basic principles of image processing technology, an image is composed of many pixels (the smallest unit of an image), and pixels contain information about color and brightness. Different images have different pixel values ​​and color distributions, which directly lead to differences between images in visual perception. After the face image is forged, the pixel association and color distribution inside the image are actually destroyed, resulting in the natural formation of intrusion areas and original areas in the forged face image that are isolated from each other but difficult to distinguish with the naked eye. Experiments have shown that the use of pixel association and distribution inconsistency to model and analyze the UNET series segmentation model can effectively locate the inconsistent areas of the forged face image with the original image.

[0052] S3. Perform interpretation and analysis on the face image based on face disorder to obtain interpretation and analysis result III.

[0053] The main process of interpretation and analysis based on facial disharmony is as follows Figure 6As shown in the figure, the face image (or the mask added in the previous step) is input into the trained mask2former network to obtain the defect area that is inconsistent with the normal face. From the interpretation and analysis results, it can be seen that the earring area and the right hair in the original image contain areas that are inconsistent with the normal face. The analysis results are post-processed to obtain a binary mask map. The mask2former network needs to be trained with specific data, and marking the training data only needs to mark the defect areas that are identifiable by the naked eye and inconsistent with the normal face.

[0054] Whether editing or synthesizing face images, or purely generating face images, there are basically certain non-coordinated areas that do not conform to human vision, that is, the defects of the forged face images. Transformer uses the self-attention mechanism to realize the encoding and representation learning of the input sequence, which can capture the dependencies between different positions in the sequence and realize context perception. Using the self-attention mechanism to find the defects in the forged face images is more in line with the human eye's behavior of viewing something, and the addition of a mask increases the model's understanding ability, making the algorithm better adapted to the scene. Experiments have shown that the interpretation and analysis method of the Transformer series model can effectively locate the defects of facial disharmony.

[0055] S4. Perform interpretation and analysis on the face image based on the deep algorithm feature vector to obtain interpretation and analysis result IV.

[0056] The main process of interpretation and analysis based on deep algorithm feature vector is as follows Figure 7 As shown in the figure, the face image (or the mask added in the previous step) is input into the convolutional neural network to obtain the high-dimensional features of the face, and the features are sparsely mapped into 2560-dimensional vectors. When performing a visual display, vectors that tend to be real features are marked in blue, and vectors that tend to be fake features are marked in red (experiments have shown that real feature vectors present positive values, that is, the y-axis value is greater than 0, and fake feature vectors present negative values, that is, the y-axis value is less than 0). Then the values ​​are sorted and the final feature vector statistical graph is output.

[0057] The difference between real images and forged images can be characterized in the form of high-dimensional features. Powerful and accurate feature extractors and a variety of forged data are essential. Therefore, using a convolutional series model as a forged feature extraction network and applying millions of data to improve the generalization performance of the model is the key to achieving the difference between true and false facial images. The high-dimensional features extracted by the model seriously affect people's sensory judgment and in-depth understanding due to the lack of visibility of their dimensions. For this reason, this example sparses the high-dimensional features into 2560-dimensional (or other multi-dimensional) vectors for visualization, marks them with different colors, and then sorts them numerically. Experiments have shown that this method can effectively show the difference between true and false facial images.

[0058] S5. Based on the interpretation and analysis results I, II, III, and IV, the interpretation and analysis results of the forged face are obtained. A single interpretation model cannot perform all-round and hierarchical interpretation and analysis of forged face images. The interpretation and analysis system of this embodiment gradually analyzes the above four key points. Each method covers an application scenario of forged face images and explains the forgery principle in depth, and finally forms a report form that decomposes from shallow to deep and layer by layer.

[0059] Embodiment 2: This embodiment is a forged face interpretation and analysis device, including: an image acquisition module, interpretation and analysis modules I, II, III, IV, and a comprehensive analysis module, etc.

[0060] In this example, the image acquisition module is used to acquire the face image to be interpreted and analyzed; the interpretation and analysis module I is used to interpret and analyze the face image based on the image compression difference, and obtain the interpretation and analysis result I; the interpretation and analysis module II is used to interpret and analyze the face image based on pixel association and distribution inconsistency, and obtain the interpretation and analysis result II; the interpretation and analysis module III is used to interpret and analyze the face image based on face disorder, and obtain the interpretation and analysis result III; the interpretation and analysis module IV is used to interpret and analyze the face image based on the deep algorithm feature vector, and obtain the interpretation and analysis result IV; the comprehensive analysis module is used to obtain the interpretation and analysis result of the forged face based on the interpretation and analysis results I, II, III, and IV.

[0061] Embodiment 3: This embodiment is a storage medium on which a computer program that can be executed by a processor is stored. When the computer program is executed, the steps of the forged face interpretation and analysis method in Embodiment 1 are implemented.

[0062] Embodiment 4: This embodiment is a forged face interpretation and analysis device, which has a memory and a processor. The memory stores a computer program that can be executed by the processor. When the computer program is executed, the steps of the forged face interpretation and analysis method in Embodiment 1 are implemented.

Claims

1. A method for explaining and analyzing forged faces, It is characterized in that include: Obtaining a facial image to be interpreted and analyzed; Based on the difference of image compression, the facial image is interpreted and analyzed, and the interpretation and analysis result I is obtained; Based on pixel association and distribution inconsistency, the face image is interpreted and analyzed, and the interpretation and analysis results II are obtained; Based on face disorder, the face image is interpreted and analyzed to obtain interpretation and analysis results III; Based on the deep algorithm feature vector, the face image is interpreted and analyzed to obtain the interpretation and analysis result IV; Based on the explanation and analysis results Ⅰ, Ⅱ, Ⅲ, and Ⅳ, the explanation and analysis results of the forged face are obtained.

2. The forged face interpretation and analysis method according to claim 1, It is characterized in that The facial image is interpreted and analyzed based on the image compression difference to obtain the interpretation and analysis result I, including: The Laplace operator is applied to the face image to extract the image gradient, and the morphological correction is performed to obtain the interpretation and analysis results.

3. The forged face interpretation and analysis method according to claim 1, It is characterized in that The facial image is interpreted and analyzed based on pixel association and distribution inconsistency to obtain interpretation and analysis result II, including: The face image is input into the trained unet++ network to obtain the foreign intrusion area with inconsistent pixel association and distribution in the face image.

4. The forged face interpretation and analysis method according to claim 1, It is characterized in that The facial image is interpreted and analyzed based on facial disorder to obtain interpretation and analysis result III, including: The face image is input into the trained mask2former network to obtain the defective area that is inconsistent with the normal face.

5. The forged face interpretation and analysis method according to claim 1, It is characterized in that The facial image is interpreted and analyzed based on the deep algorithm feature vector to obtain the interpretation and analysis result IV, including: Input the face image into the convolutional neural network to obtain the high-dimensional features of the face and sparse the features into multi-dimensional vectors; When performing a visual display, vectors that tend to be real features are marked in blue, and vectors that tend to be fake features are marked in red. Then the values ​​are sorted and the final feature vector statistics graph is output.

6. A forged face interpretation and analysis device, It is characterized in that include: An image acquisition module, used to acquire a face image to be interpreted and analyzed; The explanation and analysis module I is used to explain and analyze the face image based on the image compression difference to obtain the explanation and analysis result I; Explanation and analysis module II, used to interpret and analyze the face image based on pixel association and distribution inconsistency, and obtain explanation and analysis result II; The interpretation and analysis module III is used to interpret and analyze the face image based on the face disorder to obtain the interpretation and analysis result III; The interpretation and analysis module IV is used to interpret and analyze the face image based on the deep algorithm feature vector to obtain the interpretation and analysis result IV; The comprehensive analysis module is used to obtain the forged face interpretation analysis results based on the interpretation analysis results I, II, III, and IV.

7. A storage medium having stored thereon a computer program executable by a processor, Features: When the computer program is executed, the steps of the forged face interpretation and analysis method according to any one of claims 1 to 5 are implemented.

8. A forged face interpretation and analysis device, comprising a memory and a processor, wherein the memory stores a computer program executable by the processor, Features: When the computer program is executed, the steps of the forged face interpretation and analysis method according to any one of claims 1 to 5 are implemented.