False image detection method and device based on image self-fusion, equipment and medium

Through image self-fusion technology, natural images and facial marks are processed, masked images are generated, and target images are self-fusion, and deep forgery detection models are trained, which solves the problem of poor detection performance in the face of unknown tampering technology, and improves the problem of the existing technology's poor detection performance in the face of unknown tampering technology. Generalization ability and robustness of fake image detection.

CN119091516APending Publication Date: 2024-12-06HUNAN INST OF ADVANCED TECH
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410887218.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-03
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing facial tampering and forgery detection methods are poor in the face of unknown tampering technology, and there are problems with overfitting, resulting in a degradation of detection performance and it is difficult to effectively identify forgery images in practical applications.

Method used

Using a depth forged image detection method based on image self-fusion, image processing, size scaling, translation transformation, enhancement and image block pooling are performed by acquiring natural images and facial marks, masked images are generated, and target images and transformed images are combined. Self-fusion, train deep forged detection models to improve generalization capabilities.

Benefits of technology

It significantly improves the generalization ability and robustness of deep forged image detection, can more effectively identify unknown tampering technologies, and improves the reliability and practicality of image forged detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091516A_ABST
    Figure CN119091516A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of forged image detection, and relates to a forged image detection method and device based on image self-fusion, equipment and a medium. The method comprises the steps of obtaining a natural image, and extracting a face mark and a face contour; processing the facial mark and the facial contour of the natural image by adopting an image processing method to obtain a target image and a source image; performing size scaling and translation transformation processing on the source image to obtain a transformed image; performing enhancement processing on the transformed image to obtain an enhanced image; performing image block-based pooling processing on the enhanced image to obtain a mask image; carrying out image self-fusion on the target image, the transformation image and the mask image to obtain a fused image; and taking the natural image and the fused image as an image pair, inputting a plurality of image pairs into the initial model for training until a preset condition is met, and obtaining a detection model to detect a forged image. According to the invention, the generalization capability and robustness of deep counterfeit image detection can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of forged image detection, and in particular to a forged image detection method, device, equipment and medium based on image self-fusion. Background Art

[0002] With the development of the times and the advancement of technology, research on facial tampering technology is changing with each passing day. These studies allow attackers to easily modify facial features in existing images to generate new forged images. This ability not only changes identity information and facial attributes, but it also becomes increasingly difficult for the human eye to discern whether an image has been tampered with. This perceptual challenge presents significant opportunities for malicious users, who can use these technologies to commit identity fraud, create false evidence, or impersonate criminal activities. Such abuse poses a serious threat to social security and personal privacy.

[0003] Therefore, in the current environment, there is an urgent need to develop effective anti-tampering detection technologies to meet this challenge.

[0004] In the prior art, in the field of facial tampering and forgery detection, most existing research works adopt supervised learning methods. These methods can achieve up to 98% detection accuracy on a specific test set by leveraging known facial manipulation techniques.

[0005] However, when these methods are applied to the real world, they often suffer from overfitting problems, which can lead to a significant drop in detection performance, and in some cases even worse than random guessing. Specifically, these methods encounter overfitting problems because they directly use existing tampering techniques to train deep fake detection models, which can achieve good results on known tampering techniques, but damages the model's generalization ability when faced with unknown tampering and forgery methods; in other words, these models may perform well on data of specific tampering types that they have seen during training, but when they encounter new, unknown tampering techniques, their performance will be greatly reduced. Summary of the invention

[0006] Based on this, it is necessary to provide a forged image detection method, device, equipment and medium based on image self-fusion to address the above-mentioned technical problems, which can improve the generalization ability and robustness of deep forged image detection.

[0007] The forged image detection method based on image self-fusion includes:

[0008] Get natural images and extract facial landmarks and facial contours;

[0009] The facial markers and facial contours of the natural image are processed by image processing methods to obtain the target image and the source image; the source image is scaled and translated to obtain the transformed image;

[0010] Performing enhancement processing on the transformed image to obtain an enhanced image; performing image block-based pooling processing on the enhanced image to obtain a mask image;

[0011] The target image, the transformed image and the mask image are self-fused to obtain a fused image;

[0012] Taking a natural image and a fused image as an image pair, multiple image pairs are input into the initial model for training until the preset conditions are met, and a detection model is obtained to detect forged images.

[0013] In one embodiment, the enhanced image is subjected to image block-based pooling processing to obtain a mask image, including:

[0014] The enhanced image is divided into multiple image blocks according to the spatial position of the pixels, and the grayscale value in each image block is unified to the average grayscale value of the corresponding image block to obtain a mask image after image block pooling processing.

[0015] In one embodiment, the enhanced image is divided into a plurality of image blocks according to the spatial positions of the pixels, including:

[0016]

[0017] The grayscale value in each image block is unified as the average grayscale value of the corresponding image block, including:

[0018] M PB (i·s:(i+1)·s,j·s:(j+1)·s)=μ i,j

[0019] In the formula, μ i,j is the average pixel value of the image block in the i-th row and j-th column of the grayscale value mask M of the face, i is the row index of the image block in the grayscale value mask M, j is the column index of the image block in the grayscale value mask M, s is the side length of the square image block, x is the relative horizontal coordinate inside the image block, y is the relative vertical coordinate inside the image block, M is the facial grayscale value mask created by the mask generator, M PB is the mask image.

[0020] In one embodiment, the objective function for training the initial model includes:

[0021]

[0022] In the formula, is the cross entropy loss function used in model training, N is the total number of samples used in the training process, c is the true category label of the kth training image, is the dataset of the entire training image, is the predicted category label of the model for the kth training image.

[0023] In one embodiment, obtaining a natural image and extracting facial landmarks and facial contours includes:

[0024] Get natural images and extract facial landmarks and facial boundaries;

[0025] A preset margin is added to the facial boundary to obtain the facial contour.

[0026] In one embodiment, an image processing method is used to process facial markers and facial contours of a natural image to obtain a target image and a source image, including:

[0027] At the same time, multiple image processing methods are used to process the facial landmarks and facial contours of natural images to distinguish colors and frequencies to obtain the target image and source image.

[0028] In one embodiment, performing enhancement processing on the transformed image to obtain an enhanced image includes:

[0029] Performing marker detection on the transformed image to obtain a marker image;

[0030] For the landmark image, the convex hull method is used to segment the inner and outer faces to obtain the segmented image;

[0031] The segmented image is enhanced by mask enhancement to obtain an enhanced image.

[0032] A forged image detection device based on image self-fusion, comprising:

[0033] An acquisition module, used to acquire natural images and extract facial landmarks and facial contours;

[0034] A processing module is used to process the facial markers and facial contours of the natural image using an image processing method to obtain a target image and a source image; and to perform size scaling and translation transformation processing on the source image to obtain a transformed image;

[0035] A pooling module is used to perform enhancement processing on the transformed image to obtain an enhanced image; and perform image block-based pooling processing on the enhanced image to obtain a mask image;

[0036] A fusion module is used to self-fuse the target image, the transformed image and the mask image to obtain a fused image;

[0037] The detection module is used to take a natural image and a fused image as an image pair, input multiple image pairs into an initial model for training, and obtain a detection model when preset conditions are met to detect forged images.

[0038] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0039] Get natural images and extract facial landmarks and facial contours;

[0040] The facial markers and facial contours of the natural image are processed by image processing methods to obtain the target image and the source image; the source image is scaled and translated to obtain the transformed image;

[0041] Performing enhancement processing on the transformed image to obtain an enhanced image; performing image block-based pooling processing on the enhanced image to obtain a mask image;

[0042] The target image, the transformed image and the mask image are self-fused to obtain a fused image;

[0043] Taking a natural image and a fused image as an image pair, multiple image pairs are input into the initial model for training until the preset conditions are met, and a detection model is obtained to detect forged images.

[0044] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0045] Get natural images and extract facial landmarks and facial contours;

[0046] The facial markers and facial contours of the natural image are processed by image processing methods to obtain the target image and the source image; the source image is scaled and translated to obtain the transformed image;

[0047] Performing enhancement processing on the transformed image to obtain an enhanced image; performing image block-based pooling processing on the enhanced image to obtain a mask image;

[0048] The target image, the transformed image and the mask image are self-fused to obtain a fused image;

[0049] Taking a natural image and a fused image as an image pair, multiple image pairs are input into the initial model for training until the preset conditions are met, and a detection model is obtained to detect forged images.

[0050] The above-mentioned forged image detection method, device, equipment and medium based on image self-fusion is a highly generalized deep forged image detection method based on image block self-fusion, which performs image self-fusion on the target image, the transformed image and the mask image to simulate the characteristics and inherent forgery traces of the deep forged image, and uses the original natural image and the obtained fused image seat sample pair to train a deep forged image detection model with high generalization performance, providing a richer information source for the detection model and providing more challenging training data to perform image block-level self-fusion on the training image data by utilizing the characteristics of the input data of the ViT series model. Compared with the existing optimal methods, the method of the present application has achieved impressive detection performance when facing unknown deep forgery methods. The detection model trained by the method of the present application can learn highly generalized representations in forged image data, significantly improving its generalization ability and robustness in detecting deep forged images (tampered images), and effectively improving the reliability and practicality of image forgery detection in practical application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a diagram of an application scenario of a forged image detection method based on image self-fusion in one embodiment;

[0052] Figure 2 is a schematic flow chart of a forged image detection method based on image self-fusion in one embodiment;

[0053] Figure 3 A schematic diagram of a training process for deep fake detection in one embodiment;

[0054] Figure 4 is a schematic diagram of the architecture of a forged image detection method based on image self-fusion in one embodiment;

[0055] Figure 5 Schematic diagram of inherent traces existing in a deep fake image in an embodiment, wherein (a) is natural image one, (b) is a deep fake image with a flag bit of natural image one changed, (c) is natural image two, (d) is a deep fake image with a boundary of natural image two changed, (e) is natural image three, (f) is a deep fake image with a color of natural image three changed, (g) is natural image four, and (h) is a deep fake image with a frequency of natural image four changed;

[0056] Figure 6 is a structural block diagram of a forged image detection device based on image self-fusion in one embodiment;

[0057] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0059] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0060] In addition, the descriptions of "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "multiple groups" means at least two groups, such as two groups, three groups, etc., unless otherwise clearly and specifically defined.

[0061] In this application, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0062] In addition, the technical solutions between the various embodiments of the present application can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0063] The forged image detection method based on image self-fusion provided in this application can be applied to Figure 1 In the application scenario diagram shown, the terminal 102 communicates with the server 104 through the network, and the terminal 102 may include but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 104 may be various portal websites, servers corresponding to the backend of the work system, etc.

[0064] This application provides a forged image detection method based on image self-fusion, such as Figure 2 The flowchart shown in FIG. 1 shows a method for applying the method to Figure 1 The following is an example of a terminal in the example, including:

[0065] Step 202: Acquire a natural image and extract facial landmarks and facial contours.

[0066] Specifically:

[0067] A natural image is acquired, and facial landmarks and facial boundaries are extracted; a margin of a preset range is added to the facial boundary to obtain a facial contour.

[0068] In this step, when the acquired file is an image file, multiple natural images can be directly obtained. When the acquired file is a video file, 32 frames are extracted from the video file to obtain the natural image.

[0069] For natural images, Dlib is used to locate 81 facial landmarks and RetinaFace is used to detect facial boundaries. During the training phase, a 4-20% margin is randomly added to the facial boundary area. During the inference phase, the margin is fixed to 12.5% ​​to expand the range of the image to parts such as ears and hair, thereby improving the diversity of the image.

[0070] Step 204, using an image processing method to process the facial markers and facial contours of the natural image to obtain a target image and a source image; performing size scaling and translation transformation processing on the source image to obtain a transformed image.

[0071] Specifically:

[0072] At the same time, a variety of image processing methods are used to process the facial markers and facial contours of natural images to distinguish colors and frequencies, thereby obtaining target images and source images; the source images are scaled and translated to obtain transformed images.

[0073] In this step, the image processing method includes data enhancement techniques such as RGBShift, HueSaturationValue, RandomBrightnessContrast, ImageCompression, Downscale and Sharpen to transform the color and frequency of natural images, that is, real facial images, so as to simulate the color and frequency difference artifacts of the forged image, and the obtained target image and the source image differ only in color and frequency.

[0074] The resizing and translation transformation processing can simulate the mismatch of the facial landmarks and the inner and outer face boundaries of the forged image, and there are slight differences in the facial positions between the transformed image and the target image.

[0075] Step 206: Perform enhancement processing on the transformed image to obtain an enhanced image; perform image block-based pooling processing on the enhanced image to obtain a mask image.

[0076] Specifically:

[0077] The transformed image is detected by the landmark position to obtain the landmark image; the landmark image is segmented by the convex hull method to obtain the segmented image; the segmented image is enhanced by mask enhancement to obtain the enhanced image. The enhanced image is divided into multiple image blocks according to the spatial position of the pixel points, and the grayscale value in each image block is unified to the average grayscale value of the corresponding image block to obtain the mask image after the image block pooling process.

[0078] More specifically:

[0079] Applying facial region key point detector to the transformed image to obtain the landmark image;

[0080] For the landmark image, the convex hull method based on the predicted facial landmarks is used to perform inner and outer face segmentation, and the grayscale image mask is initialized with the convex hull boundary to obtain the segmented image;

[0081] For the segmented image, elastic deformation is used to adjust the shape and mixing ratio of the grayscale mask to enhance the diversity of the fused mask, and resizing and translation transformation are performed to obtain an enhanced image.

[0082] Divide the enhanced image into multiple image blocks according to the spatial position of the pixels:

[0083]

[0084] The grayscale value in each image block is unified to the average grayscale value of the corresponding image block:

[0085] M PB (i·s:(i+1)·s,j·s:(j+1)·s)=μ i,j

[0086] In the formula, μ i,j is the average pixel value of the image block in the i-th row and j-th column of the grayscale value mask M of the face, i is the row index of the image block in the grayscale value mask M, j is the column index of the image block in the grayscale value mask M, s is the side length of the square image block, x is the relative horizontal coordinate inside the image block, y is the relative vertical coordinate inside the image block, M is the facial grayscale value mask created by the mask generator, M PB is the mask image, that is, the grayscale value mask of the image block obtained by performing image block-based pooling processing on M;

[0087] Finally, the mask image M is obtained after image block pooling.PB .

[0088] In this step, the enhancement processing is performed based on the grayscale mask generation technology of the internal and external faces of the image block; the pooling processing based on the image block level is performed according to the spatial position of each grayscale pixel to fully utilize the characteristics of the Transformer series model architecture.

[0089] It should be noted that the facial region key point detector and the convex hull method are both existing technologies.

[0090] Step 208: self-fuse the target image, the transformed image and the mask image to obtain a fused image.

[0091] In this step, the fused image is a fusion of three images, which can produce overlapping artifact traces.

[0092] Using the patch-based mask image M PB Transform image I S and the target image I T Fusion is performed to generate forged samples containing multiple forged trace types and different fusion degrees, namely, fused image I SBP :

[0093] I SBP =I S ⊙M PB +I T ⊙(1-M PB )

[0094] The forged sample, that is, the fusion image, has a constant fusion degree within the image block, but different image blocks have different fusion degrees, and can simulate multiple forged fusion degrees in a finer granularity in the same image at the same time.

[0095] Step 210, taking the natural image and the fused image as an image pair, multiple image pairs are input into the initial model for training until the preset conditions are met, and a detection model is obtained to detect forged images.

[0096] Specifically:

[0097] A natural image and a fused image are taken as an image pair. The fused image contains forgery traces of different granularity and degree. Multiple image pairs are input into the initial model as training samples for deep forgery detection training. When the preset conditions are met, a deep forgery detection model is obtained to detect deep forgery images.

[0098] Among them, the initial model is trained using the cross entropy loss function as the objective function:

[0099]

[0100] In the formula, is the cross entropy loss function used in model training, that is, the objective function, N is the total number of samples used in the training process, c is the true category label of the kth training image, is the dataset of the entire training image, is the predicted category label of the model for the kth training image.

[0101] In this step, the training process of deep fake detection is as follows Figure 3 As shown, deep fake detection training enables the detection model to learn highly generalized representations in forged images, and ultimately obtains a deep fake detection model with high generalization performance, which can improve the generalization performance and robustness of various deep fake methods in open environments.

[0102] Note: The initial model may adopt the ViT (Vision Transformer) series model.

[0103] In this embodiment, if Figure 4 The architecture diagram shown in the figure shows that the natural image, that is, the real image, is input into the source-target generator (STG) and the image block-based mask generator (PBMG); STG applies a general image processing method to the real image to generate the target image and the source image; at the same time, PBMG generates grayscale value masks of the inner face and the outer face based on the real facial landmarks, and these masks are further combined through the average pooling at the image block level to generate a more fine-grained fusion mask; the fusion mask is applied to fuse the target image and the source image at the image block level to generate a fused image that is self-fused at the image block level. Finally, these fused images, together with the real image, constitute the sample pairs for model training.

[0104] This application aims to improve the generalization performance of the deep fake detection model when facing unknown counterfeiting methods. In order to cultivate a more universal and robust true and false image recognition system, this application relies on the inherent defects of deep fake technology and creatively generates a series of synthetic sample pairs. These samples can cover a variety of common and varying degrees of counterfeiting traces. Specifically: using an image block-based mask strategy, the source image and the target image are cleverly merged to produce a counterfeit image. This mask-based image synthesis process is performed at the image block level and can effectively simulate a series of hidden counterfeit features, including but not limited to: inconsistent boundaries, differences in facial skin color, frequency inconsistency, and facial feature misalignment. Problems such as Figure 5 Inherent traces present in the deepfake images shown.

[0105] The above-mentioned forged image detection method based on image self-fusion is a highly generalized deep forged image detection method based on image block self-fusion, which performs image self-fusion on the target image, the transformed image and the mask image to simulate the characteristics and inherent forgery traces of deep forged images, and uses the original natural image and the obtained fused image seat sample pairs to train a deep forged image detection model with high generalization performance, providing a richer information source for the detection model and providing more challenging training data to perform image block-level self-fusion on the training image data by utilizing the characteristics of the input data of the ViT series model. Compared with the existing optimal methods, the method of the present application has achieved impressive detection performance when facing unknown deep forgery methods. The detection model trained by the method of the present application can learn highly generalized representations in forged image data, significantly improving its generalization ability and robustness in detecting deep forged images (tampered images), and effectively improving the reliability and practicality of image forgery detection in actual application scenarios.

[0106] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0107] The present application also provides a device, such as Figure 6 As shown, in one embodiment, it includes: an acquisition module 602, a processing module 604, a pooling module 606, a fusion module 608 and a detection module 610, wherein:

[0108] An acquisition module 602 is used to acquire natural images and extract facial landmarks and facial contours;

[0109] The processing module 604 is used to process the facial marks and facial contours of the natural image using an image processing method to obtain a target image and a source image; and to perform size scaling and translation transformation processing on the source image to obtain a transformed image;

[0110] The pooling module 606 is used to perform enhancement processing on the transformed image to obtain an enhanced image; perform image block-based pooling processing on the enhanced image to obtain a mask image;

[0111] A fusion module 608 is used to perform self-fusion on the target image, the transformed image and the mask image to obtain a fused image;

[0112] The detection module 610 is used to take a natural image and a fused image as an image pair, input multiple image pairs into an initial model for training, and obtain a detection model to detect forged images when preset conditions are met.

[0113] For the specific definition of the forged image detection device based on image self-fusion, please refer to the definition of the forged image detection method based on image self-fusion above, which will not be repeated here. Each module in the above device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0114] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a forged image detection method based on image self-fusion is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0115] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0116] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0117] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0118] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0119] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0120] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A forged image detection method based on image self-fusion, characterized in that: include: Get natural images and extract facial landmarks and facial contours; The facial markers and facial contours of the natural image are processed by image processing methods to obtain the target image and the source image; the source image is scaled and translated to obtain the transformed image; Performing enhancement processing on the transformed image to obtain an enhanced image; Perform image block-based pooling processing on the enhanced image to obtain a mask image; The target image, the transformed image and the mask image are self-fused to obtain a fused image; Taking a natural image and a fused image as an image pair, multiple image pairs are input into the initial model for training until the preset conditions are met, and a detection model is obtained to detect forged images.

2. The forged image detection method based on image self-fusion according to claim 1 is characterized in that: The enhanced image is processed based on image block pooling to obtain a mask image, including: The enhanced image is divided into multiple image blocks according to the spatial position of the pixels, and the grayscale value in each image block is unified to the average grayscale value of the corresponding image block to obtain a mask image after image block pooling processing.

3. The forged image detection method based on image self-fusion according to claim 2 is characterized in that: The enhanced image is divided into multiple image blocks according to the spatial position of the pixels, including: The grayscale value in each image block is unified as the average grayscale value of the corresponding image block, including: M PB (i·s:(i+1)·s,j·s:(j+1)·s)=μ i,j In the formula, μ i,j is the average pixel value of the image block in the i-th row and j-th column of the grayscale value mask M of the face, i is the row index of the image block in the grayscale value mask M, j is the column index of the image block in the grayscale value mask M, s is the side length of the square image block, x is the relative horizontal coordinate inside the image block, y is the relative vertical coordinate inside the image block, M is the facial grayscale value mask created by the mask generator, M PB is the mask image.

4. The forged image detection method based on image self-fusion according to any one of claims 1 to 3, characterized in that: The objective function for training the initial model includes: In the formula, is the cross entropy loss function used in model training, N is the total number of samples used in the training process, c is the true category label of the kth training image, is the dataset of the entire training image, is the predicted category label of the model for the kth training image.

5. The forged image detection method based on image self-fusion according to any one of claims 1 to 3, characterized in that: Take natural images and extract facial landmarks and facial contours, including: Get natural images and extract facial landmarks and facial boundaries; A preset margin is added to the facial boundary to obtain the facial contour.

6. The forged image detection method based on image self-fusion according to any one of claims 1 to 3, characterized in that: The facial markers and facial contours of the natural image are processed by image processing methods to obtain the target image and the source image, including: At the same time, multiple image processing methods are used to process the facial landmarks and facial contours of natural images to distinguish colors and frequencies to obtain the target image and source image.

7. The forged image detection method based on image self-fusion according to any one of claims 1 to 3, characterized in that: The transformed image is enhanced to obtain an enhanced image, including: Performing marker detection on the transformed image to obtain a marker image; For the landmark image, the convex hull method is used to segment the inner and outer faces to obtain the segmented image; The segmented image is enhanced by mask enhancement to obtain an enhanced image.

8. A forged image detection device based on image self-fusion, characterized in that: include: An acquisition module, used to acquire natural images and extract facial landmarks and facial contours; A processing module is used to process the facial markers and facial contours of the natural image using an image processing method to obtain a target image and a source image; and to perform size scaling and translation transformation processing on the source image to obtain a transformed image; A pooling module is used to enhance the transformed image to obtain an enhanced image; Perform image block-based pooling processing on the enhanced image to obtain a mask image; A fusion module is used to self-fuse the target image, the transformed image and the mask image to obtain a fused image; The detection module is used to take a natural image and a fused image as an image pair, input multiple image pairs into an initial model for training, and obtain a detection model when preset conditions are met to detect forged images.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Image forgery detection method, system and device based on pixel difference perception

    CN120355949A

  • Deep forgery detection method and device based on mixed training

    CN120375449A