Image authenticity detection method and image authenticity detection model training method

Abstract

The application relates to the field of artificial intelligence, in particular to an image authenticity detection method. The method comprises the following steps: obtaining a frequency domain missing image according to part of frequency domain information in a to-be-detected image, and obtaining a color missing image according to part of color information; generating a frequency domain reconstruction image according to first image features in the frequency domain missing image, and generating a color reconstruction image according to second image features in the color missing image; determining a frequency domain difference mask according to the difference between the frequency domain reconstruction image and the to-be-detected image, processing the first image features through the frequency domain difference mask, and obtaining frequency domain suspected forgery features; determining a color difference mask according to the difference between the color reconstruction image and the to-be-detected image, processing the second image features through the color difference mask, and obtaining color suspected forgery features; and comprehensively determining an authenticity detection result of the to-be-detected image by combining the frequency domain suspected forgery features and the color suspected forgery features. The method can improve the accuracy of the authenticity detection result.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image authenticity detection method and an image authenticity detection model training method. Background Technology

[0002] With the development of image editing technology, people can freely edit the content of images, leading to a surge in high-quality, deceptively realistic forgeries. This seriously impacts the security of currently deployed image recognition systems. Therefore, detecting whether input images have been edited is increasingly becoming a crucial aspect of cybersecurity.

[0003] Currently, machine learning models are primarily used to detect specific forgery traces in images, and the detection of these traces is used to determine whether an image is fake. For example, the authenticity of an image can be judged by detecting the degree of matching between global and local illumination. However, judging whether an image is fake by detecting specific forgery traces can easily lead to low generalization of machine learning models, resulting in low accuracy of the authenticity detection results output by machine learning models with low generalization. Summary of the Invention

[0004] Therefore, it is necessary to provide an image authenticity detection method, apparatus, computer equipment, and storage medium that can improve the detection accuracy in addressing the aforementioned technical problems.

[0005] An image authenticity detection method, the method comprising:

[0006] The image to be detected is acquired, and a frequency domain missing map is obtained based on some frequency domain information in the image to be detected, and a color missing map is obtained based on some color information in the image to be detected.

[0007] A frequency domain reconstruction map is generated based on the first image features in the frequency domain missing map, and a color reconstruction map is generated based on the second image features in the color missing map;

[0008] Based on the difference between the frequency domain reconstructed image and the image to be detected, a frequency domain difference mask is determined, and the first image features are subjected to self-attention processing using the frequency domain difference mask to obtain frequency domain suspected forgery features.

[0009] Based on the difference between the color reconstruction map and the image to be detected, a color difference mask is determined, and the second image features are subjected to self-attention processing using the color difference mask to obtain color suspected forgery features;

[0010] By combining the frequency domain suspected forgery features and the color suspected forgery features, the authenticity detection result of the image to be detected is determined.

[0011] In one embodiment, acquiring the image to be detected includes:

[0012] Acquire the video to be detected, and extract multiple video frames from the video to be detected based on the sampling frequency;

[0013] Each extracted video frame is used as the image to be detected;

[0014] The method further includes:

[0015] The authenticity detection result of the video to be detected is determined by combining the authenticity detection results corresponding to each of the images to be detected.

[0016] An image authenticity detection device, the device comprising:

[0017] The image reconstruction generation module is used to acquire an image to be detected, and to obtain a frequency domain missing map based on some frequency domain information in the image to be detected, and to obtain a color missing map based on some color information in the image to be detected; to generate a frequency domain reconstruction map based on a first image feature in the frequency domain missing map, and to generate a color reconstruction map based on a second image feature in the color missing map.

[0018] The suspected feature determination module is used to determine a frequency domain difference mask based on the difference between the frequency domain reconstructed image and the image to be detected, and to perform self-attention processing on the first image features through the frequency domain difference mask to obtain frequency domain suspected forgery features; and to determine a color difference mask based on the difference between the color reconstructed image and the image to be detected, and to perform self-attention processing on the second image features through the color difference mask to obtain color suspected forgery features.

[0019] The result output module is used to combine the frequency domain suspected forgery features and the color suspected forgery features to determine the authenticity detection result of the image to be detected.

[0020] In one embodiment, the reconstructed image generation module is further configured to acquire the video to be detected and extract multiple video frames from the video to be detected according to the sampling frequency; each extracted video frame is used as an image to be detected. The image authenticity detection device is further configured to integrate the authenticity detection results corresponding to each image to be detected to determine the authenticity detection result of the video to be detected.

[0021] In one embodiment, the reconstructed image generation module further includes a missing map generation module, used to convert the image to be detected from the image spatial domain to the frequency domain to obtain a frequency domain image; filter frequency domain information in the frequency domain image whose spatial frequency is greater than a preset frequency threshold to obtain target frequency domain information in the image to be detected; and convert the target frequency domain information from the frequency domain to the image spatial domain to obtain a frequency domain missing map.

[0022] In one embodiment, the missing image generation module is further configured to segment the image to be detected to obtain an image grid comprising multiple image slices; and convert at least one image slice in the image grid into a grayscale image to obtain a corresponding color missing image.

[0023] In one embodiment, the suspected feature determination module further includes a frequency domain mask generation module, used to determine a first image difference between the frequency domain reconstructed image and the image to be detected, and to perform convolution and activation processing on the first image difference to obtain a frequency domain difference mask; and to apply the frequency domain difference mask to the first image features through a self-attention mechanism to obtain frequency domain suspected forgery features.

[0024] In one embodiment, the frequency domain mask generation module is further configured to determine the attention weight corresponding to each first feature element in the first image feature according to the frequency domain difference mask; multiply each first feature element by its corresponding attention weight to obtain the frequency domain fusion feature; and combine the frequency domain fusion feature and the first image feature to obtain the frequency domain suspected forgery feature.

[0025] In one embodiment, the suspected feature determination module further includes a color mask generation module, used to determine a second image difference between the color reconstruction image and the image to be detected, and to perform convolution and activation processing on the second image difference to obtain a color difference mask; and to apply the color difference mask to the second image features through a self-attention mechanism to obtain color suspected forgery features.

[0026] In one embodiment, the color mask generation module is further configured to determine the attention weight corresponding to each second feature element in the second convolutional feature based on the color difference mask; multiply each second feature element by its corresponding attention weight to obtain the color fusion feature; and combine the color fusion feature and the second image feature to obtain the color suspected forgery feature.

[0027] In one embodiment, the result output module is further configured to perform feature supplementation processing on the frequency domain suspected forgery feature using the color suspected forgery feature to obtain a corresponding frequency domain supplemented feature; perform feature supplementation processing on the color suspected forgery feature using the frequency domain suspected forgery feature to obtain a corresponding color supplemented feature; and combine the frequency domain supplemented feature and the color supplemented feature to obtain the authenticity detection result of the image to be detected.

[0028] In one embodiment, the result output module further includes a feature fusion module, configured to: determine a first vector sequence corresponding to the frequency domain suspected forgery feature; perform convolution processing on the color forgery feature to obtain a color forgery convolution feature; determine the position information of each first vector element in the first vector sequence and the position information of each third feature element in the color forgery convolution feature; fuse the first vector elements and third feature elements with corresponding position information respectively to obtain a frequency domain vector fusion feature; and merge the frequency domain vector fusion feature and the frequency domain suspected forgery feature to obtain a frequency domain supplementary feature.

[0029] In one embodiment, the feature fusion module is further configured to: determine a second vector sequence corresponding to the suspected color forgery feature; perform convolution processing on the frequency domain forgery feature to obtain a frequency domain forgery convolution feature; determine the position information of each second vector element in the second vector sequence and the position information of each fourth feature element in the frequency domain forgery convolution feature; fuse the second vector elements and fourth feature elements with corresponding position information respectively to obtain a color vector fusion feature; and merge the color vector fusion feature and the suspected color forgery feature to obtain a color supplement feature.

[0030] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the image authenticity detection methods provided in the embodiments of this application.

[0031] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the image authenticity detection methods provided in the embodiments of this application.

[0032] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the image authenticity detection methods provided in the embodiments of this application.

[0033] The aforementioned image authenticity detection method, apparatus, computer equipment, storage medium, and computer program, by acquiring the image to be detected, can filter out some information in the image to obtain a frequency domain missing map and a color missing map. By generating the frequency domain missing map, image reconstruction can be performed based on the first image features in the frequency domain missing map to obtain a frequency domain reconstructed map; similarly, by generating the color missing map, image reconstruction can be performed based on the second image features in the color missing map to obtain a color reconstructed map. By generating the frequency domain reconstructed map, a frequency domain difference mask can be obtained, and thus, frequency domain suspected forgery features in the frequency domain space can be obtained based on the frequency domain difference mask; similarly, by generating the color reconstructed map, a color difference mask can be obtained, and thus, color suspected forgery features in the color space can be obtained based on the color difference mask. In this way, the authenticity detection result can be output by combining the frequency domain suspected forgery features in the frequency domain space and the color space suspected forgery features in the color space. Since the authenticity detection result is obtained by integrating frequency domain suspected forgery features in the frequency domain space and color suspected forgery features in the color space, compared with the traditional method of detecting certain specific forgery traces to detect the image to be tested, this application is not limited by specific forgery traces, thereby improving the generalization of the image authenticity detection model and thus improving the accuracy of the authenticity detection result.

[0034] A method for training an image authenticity detection model, the method comprising:

[0035] Obtain the sample image set and the sample label corresponding to each sample image in the sample image set;

[0036] A frequency domain prediction missing map is obtained based on partial frequency domain information in the sample image, and a color prediction missing map is obtained based on partial color information in the sample image;

[0037] A frequency domain prediction reconstruction map is generated based on the first prediction feature in the frequency domain prediction missing map, and a color prediction reconstruction map is generated based on the second prediction feature in the color prediction missing map.

[0038] Based on the difference between the frequency domain prediction reconstruction map and the sample image, a frequency domain prediction mask is determined, and the first prediction feature is subjected to self-attention processing through the frequency domain prediction mask to obtain a frequency domain prediction forgery feature.

[0039] Based on the difference between the color prediction reconstruction map and the sample image, a color prediction mask is determined, and the second prediction feature is subjected to self-attention processing through the color prediction mask to obtain the color prediction fake feature;

[0040] By combining the frequency domain prediction forgery features and the color prediction forgery features, the authenticity prediction result of the sample image is obtained, and the classification loss is determined based on the authenticity prediction result and the corresponding sample label.

[0041] The target loss function is determined based on the classification loss, and the image authenticity detection model is trained using the target loss function until the training stops when the stopping condition is met, thus obtaining the trained image authenticity detection model; wherein, the trained image authenticity detection model is used to perform image authenticity detection on the image to be detected, and obtain the corresponding authenticity detection results.

[0042] In one embodiment, obtaining the sample image set includes:

[0043] Acquire multiple real videos, and at least one fake video corresponding to each of the real videos;

[0044] For each of the multiple real videos, the target fake video is selected from at least one fake video corresponding to the current real video;

[0045] Video frames were extracted from each real video and each target fake video to obtain a sample image set.

[0046] An image authenticity detection model training device, the device comprising:

[0047] The prediction reconstruction map generation module is used to acquire a set of sample images and the sample label corresponding to each sample image in the set of sample images; to obtain a frequency domain prediction missing map based on some frequency domain information in the sample images, and to obtain a color prediction missing map based on some color information in the sample images; to generate a frequency domain prediction reconstruction map based on the first prediction feature in the frequency domain prediction missing map, and to generate a color prediction reconstruction map based on the second prediction feature in the color prediction missing map.

[0048] The prediction feature determination module is used to determine a frequency domain prediction mask based on the difference between the frequency domain prediction reconstruction map and the sample image, and to perform self-attention processing on the first prediction feature through the frequency domain prediction mask to obtain a frequency domain prediction forgery feature; and to determine a color prediction mask based on the difference between the color prediction reconstruction map and the sample image, and to perform self-attention processing on the second prediction feature through the color prediction mask to obtain a color prediction forgery feature.

[0049] The training module is used to integrate the frequency domain prediction forgery features and the color prediction forgery features to obtain the authenticity prediction results of the sample image, and to determine the classification loss based on the authenticity prediction results and the corresponding sample labels; to determine the target loss function based on the classification loss, and to train the image authenticity detection model through the target loss function until the training stops when the stopping condition is met, thus obtaining the trained image authenticity detection model; wherein, the trained image authenticity detection model is used to perform image authenticity detection on the image to be detected, and obtain the corresponding authenticity detection results.

[0050] In one embodiment, the prediction and reconstruction map generation module is further configured to acquire multiple real videos and at least one fake video corresponding to each real video; for each of the multiple real videos, a target fake video is selected from at least one fake video corresponding to the current real video; and video frames are extracted from each real video and each target fake video to obtain a sample image set.

[0051] In one embodiment, the image authenticity detection model training device is further configured to construct a first reconstruction loss based on the difference between the training sample and the corresponding frequency domain predicted reconstruction map when the training sample is a real image, and to construct a second reconstruction loss based on the difference between the training sample and the corresponding color predicted reconstruction map; and to determine the target loss function through the first reconstruction loss, the second reconstruction loss and the classification loss.

[0052] In one embodiment, the image authenticity detection model training device is further configured to determine a first training sample pair and a second training sample pair in the training sample set; the first training sample pair includes two training samples with the same authenticity category; the second training sample pair includes two training samples with different authenticity categories; determine a first image distance between the first predicted features corresponding to the two training samples in the first training sample pair, and a second image distance between the second predicted features corresponding to the two training samples in the first training sample pair; determine a third image distance between the first predicted features corresponding to the two training samples in the second training sample pair, and a fourth image distance between the second predicted features corresponding to the two training samples in the second training sample pair; determine a first constraint loss based on the first image distance and the third image distance; determine a second constraint loss based on the second image distance and the fourth image distance; and determine a target loss function using the first constraint loss, the second constraint loss, the first reconstruction loss, the second reconstruction loss, and the classification loss.

[0053] A computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in any of the image authenticity detection model training methods provided in the embodiments of this application.

[0054] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in any of the image authenticity detection model training methods provided in the embodiments of this application.

[0055] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the image authenticity detection model training methods provided in the embodiments of this application.

[0056] The aforementioned image authenticity detection model training method, apparatus, computer equipment, storage medium, and computer program, by acquiring sample images, can generate frequency domain prediction missing maps and color prediction missing maps corresponding to the sample images. Based on the frequency domain prediction missing maps, corresponding frequency domain prediction reconstruction maps are generated, and based on the color prediction missing maps, corresponding color prediction reconstruction maps are generated. By generating the frequency domain prediction reconstruction maps, frequency domain prediction forgery features can be obtained based on the differences between the frequency domain prediction reconstruction maps and the corresponding sample images; by generating the color prediction reconstruction maps, color prediction forgery features can be obtained based on the differences between the color prediction reconstruction maps and the corresponding sample images. Thus, by combining the frequency domain prediction forgery features in the frequency domain space and the color prediction forgery features in the color space, a authenticity prediction result can be obtained. Based on the differences between the authenticity prediction results and the corresponding sample labels, a target loss function can be determined, and the image authenticity detection model can be trained based on the target loss function to obtain a trained image authenticity detection model. Since the image authenticity detection model is trained by combining frequency domain prediction forgery features and color prediction forgery features, the trained image authenticity detection model can output more accurate frequency domain suspected forgery features and color suspected forgery features, thereby making the authenticity detection results obtained based on more accurate frequency domain suspected forgery features and color suspected forgery features more accurate. Attached Figure Description

[0057] Figure 1 This is a diagram illustrating the application environment of an image authenticity detection method in one embodiment.

[0058] Figure 2 This is a flowchart illustrating an image authenticity detection method in one embodiment;

[0059] Figure 3 This is a schematic diagram illustrating the generation of a frequency domain missing image in one embodiment;

[0060] Figure 4 This is a schematic diagram illustrating the generation of a color-deficient image in one embodiment;

[0061] Figure 5 This is a schematic diagram of the frequency domain suspected forgery features output by the first difference attention module in one embodiment;

[0062] Figure 6 This is a schematic diagram of the output of the authenticity detection results in one embodiment;

[0063] Figure 7 This is a schematic diagram of the collaborative fusion module in one embodiment;

[0064] Figure 8 This is a schematic diagram of the overall framework of an image authenticity detection model in one embodiment;

[0065] Figure 9 This is a schematic diagram of the training process for an image authenticity detection model in one embodiment;

[0066] Figure 10 This is a schematic diagram of a sample image set in one embodiment;

[0067] Figure 11 This is a flowchart illustrating an image authenticity detection method in a specific embodiment.

[0068] Figure 12 This is a flowchart illustrating the training method for an image authenticity detection model in a specific embodiment.

[0069] Figure 13 This is a structural block diagram of an image authenticity detection device in one embodiment;

[0070] Figure 14 This is a structural block diagram of an image authenticity detection model training device in one embodiment;

[0071] Figure 15 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0073] Figure 1 This is an application environment diagram illustrating an image authenticity detection method in one embodiment. (Refer to...) Figure 1The image authenticity detection method is applied to an image authenticity detection system 100. The image authenticity detection system 100 includes a terminal 102 and a server 104. Both the terminal 102 and the server 104 can be used independently to execute the image authenticity detection method provided in this embodiment. The terminal 102 and the server 104 can also be used collaboratively to execute the image authenticity detection method provided in this embodiment. Taking the collaborative execution of the image authenticity detection method provided in this embodiment by the terminal 102 as an example, the terminal 102 can acquire the image to be detected and send it to the server 104, so that the server 104 can call the image authenticity detection model to perform image authenticity detection on the image to be detected, obtain the authenticity detection result, and return the authenticity detection result to the terminal 102 for display.

[0074] The server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal 102 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, in-vehicle terminal, smart TV, etc., but is not limited to these. The terminal 102 and the server 104 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0075] This application also relates to the field of artificial intelligence (AI). AI is the theory, methods, technology, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine capable of reacting in a manner similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. Immediately, this application specifically relates to computer vision (CV) technology within the field of AI. Computer vision is a science that studies how to enable machines to "see," and through computer vision technology, liveness detection can be performed on images.

[0076] To better understand the image authenticity detection method in the embodiments of this application, the overall concept of this application is introduced below:

[0077] Existing forged images often retain traces of forgery in both the color and frequency domains. Therefore, this application proposes an image authenticity detection technique based on self-supervised reconstruction learning. This technique reconstructs partially information-lost images in both the color and frequency domains to determine the differences between the reconstructed and original images, thereby capturing comprehensive information about the original image. Since the image authenticity detection model only calculates the reconstruction loss of real sample images during training, the difference between the real image and its corresponding reconstructed image will be smaller than the difference between the forged image and its corresponding reconstructed image during model usage. Therefore, areas with larger differences indicate potential forgery traces. Based on this, the image authenticity detection model in this application includes a Difference Attention Module (DAM) and a Collaborative Fusion Module (CFM). The DAM module determines the difference between the image to be detected and its corresponding reconstructed image, and obtains a difference mask based on this difference. This difference mask is then applied to the image features used for image reconstruction using an attention mechanism, forcing the model to focus on potentially forged regions. Furthermore, in order to promote the fusion and complementarity of features extracted based on color space and frequency domain space, this application integrates information from the two spaces through a CFM module to conduct collaborative learning, uncover more comprehensive forgery traces, and thus output more accurate authenticity detection results.

[0078] In one embodiment, such as Figure 2 As shown, an image authenticity detection method is provided, and the method is illustrated by its application to a computer device, which can specifically be... Figure 1 The terminal or server in the system.

[0079] The image authenticity detection method includes the following steps:

[0080] Step S202: Obtain the image to be detected, and obtain a frequency domain missing map based on some frequency domain information in the image to be detected, and obtain a color missing map based on some color information in the image to be detected.

[0081] In this context, a frequency domain missing image refers to an image that has lost some frequency domain information. Similarly, a color missing image refers to an image that has lost some color information. By using frequency domain missing images and color missing images, the authenticity of the image to be tested can be determined.

[0082] Image authenticity detection refers to methods used in image verification and recognition scenarios to determine whether an image or video to be identified is genuine. The authenticity detection result can indicate whether the image is genuine or fake. When the result indicates the image is genuine, it can be assumed that the image content has not been edited; when the result indicates the image is fake, it can be assumed that some or all of the image content has been edited.

[0083] Specifically, since some traces of forgery may remain in both the color space and frequency domain after image content editing, the image authenticity detection model used in this application may include a frequency domain processing branch and a color processing branch. The frequency domain processing branch can generate a missing frequency domain map and a reconstructed frequency domain map based on the missing frequency domain map; the color processing branch can generate a missing color map and a reconstructed color map based on the missing color map.

[0084] When the image to be detected is acquired, the computer device can input the image into the frequency domain processing branch and the color processing branch respectively. The frequency domain processing branch randomly removes some frequency domain information from the image to be detected and performs image reconstruction based on the remaining frequency domain information to obtain a frequency domain missing map corresponding to the image to be detected. Similarly, the color processing branch can also randomly remove some color information from the image to be detected and perform image reconstruction based on the remaining color information to obtain a color missing map.

[0085] In one embodiment, a computer device may acquire a video segment for authenticity detection, randomly extract a video frame to be detected from the video segment, and use the extracted video frame to be detected as the image to be detected.

[0086] In one embodiment, the image to be detected is converted from the image spatial domain to the frequency domain to obtain a frequency domain image; frequency domain information in the frequency domain image with spatial frequencies greater than a preset frequency threshold is filtered to obtain target frequency domain information in the image to be detected; and the target frequency domain information is converted from the frequency domain to the image spatial domain to obtain a frequency domain missing map.

[0087] The image spatial domain refers to the space composed of image pixels, and is also known as the color space. The frequency domain refers to the space that describes image features using spatial frequency (i.e., wavenumber) as the independent variable.

[0088] Specifically, when the frequency domain processing branch acquires the image to be detected, it can transform the image from the image spatial domain to the frequency domain to obtain a frequency domain image. For example, the frequency domain processing branch can perform a discrete cosine transform on the image to be detected to transform it from the image spatial domain to the frequency domain. Further, the frequency domain processing branch filters some frequency domain information in the frequency domain image to obtain the target frequency domain information in the image to be detected. The frequency domain processing branch can randomly filter some frequency domain information in the frequency domain image, or it can filter some frequency domain information according to preset rules. For example, the frequency domain processing branch can filter frequency domain information with spatial frequencies higher than a preset frequency threshold, and use frequency domain information with spatial frequencies lower than or equal to the preset frequency threshold as the target frequency domain information. Further, the frequency domain processing branch transforms the target frequency domain information from the frequency domain to the image spatial domain to obtain a frequency domain missing image. For example, the frequency domain processing branch performs an inverse discrete cosine transform on the target frequency domain information to obtain a frequency domain missing map.

[0089] In one embodiment, the frequency domain processing branch can utilize Discrete Fourier Transform or Fast Fourier Transform to convert the image to be detected into a frequency domain image.

[0090] In one embodiment, reference Figure 3 The frequency domain processing branch converts the image to be detected into a frequency domain image and identifies high-frequency information in the frequency domain image whose spatial frequency is greater than a preset frequency threshold. By randomly filtering some of the high-frequency information, the remaining high-frequency information is obtained. The frequency domain processing branch combines the remaining high-frequency information with low-frequency information whose spatial frequency is less than or equal to the preset frequency threshold to obtain the target frequency domain information, and obtains the frequency domain missing image through the target frequency domain information. Figure 3 A schematic diagram of the generation of a frequency domain missing image is shown in one embodiment.

[0091] In the above embodiments, by filtering out some frequency domain information in the frequency domain image, a frequency domain missing map can be obtained based on the remaining target frequency domain information. Subsequently, image reconstruction processing can be performed based on the frequency domain missing map to obtain the corresponding frequency domain reconstructed map.

[0092] In one embodiment, obtaining a color missing map based on partial color information in the image to be detected includes: segmenting the image to be detected to obtain an image grid comprising multiple image slices; and converting at least one image slice in the image grid into a grayscale image to obtain the corresponding color missing map.

[0093] Specifically, when the color processing branch acquires the image to be detected, it can segment the image to obtain an image grid comprising multiple image slices. It is easy to understand that the color processing branch can segment the image to be detected into multiple image slices of the same size, or it can segment the image to be detected into multiple image slices of different sizes. This implementation is not limited to this. Further, the color processing branch randomly converts at least one image slice in the image grid into a grayscale image, obtaining a color-deficient image.

[0094] In one embodiment, since any color is composed of the three primary colors of red, green, and blue, the three primary colors of the pixels in the image slice can be processed by floating-point algorithms, integer methods, shift methods, or averaging methods to convert the image slice into a grayscale image.

[0095] In one embodiment, reference Figure 4 , Figure 4 A schematic diagram illustrating the generation of a color-deficient image in one embodiment is shown. After slicing, the image to be detected can be converted into an image grid comprising multiple image slices. For example, the color processing branch can divide the image to be detected into a five-row, five-column image grid. The color processing branch can then randomly filter the colors of at least one image slice in the image grid to obtain a corresponding grayscale image. Furthermore, the color processing branch can integrate all the slices in the image grid into a color-deficient image of the same size as the image to be detected.

[0096] In the above embodiments, by converting some image slices into grayscale images, some color information in the image to be detected can be filtered out, so that subsequent image reconstruction can be performed based on the color-deficient image with missing information to obtain a color reconstruction map.

[0097] Step S204: Generate a frequency domain reconstruction map based on the first image features in the frequency domain missing map, and generate a color reconstruction map based on the second image features in the color missing map.

[0098] Among them, the frequency domain reconstruction image refers to the corresponding image generated based on the features of the first image, and the color reconstruction image refers to the corresponding image generated based on the features of the second image.

[0099] Specifically, the frequency domain processing branch may further include a first encoder and a first decoder. When generating a frequency domain missing image, the first encoder encodes the frequency domain missing image to obtain a first image feature in the frequency domain missing image, and the first decoder decodes the first image feature to obtain the corresponding frequency domain reconstructed image. Similarly, the color processing branch also includes a second encoder and a second decoder. When generating a color missing image, the second encoder encodes the color missing image to obtain a second image feature, and the second decoder decodes the second image feature to obtain the corresponding color reconstructed image. Both the first encoder and the second encoder may include multiple convolutional layers.

[0100] In one embodiment, the structure of the first encoder can be the same as that of the second encoder, and the structure of the first decoder can also be the same as that of the second decoder. However, since the first encoder and the second encoder correspond to different inputs, the model parameters in the first encoder obtained after model training are not the same as those in the second encoder. Correspondingly, the model parameters in the first decoder obtained after model training are also not the same as those in the second decoder.

[0101] Step S206: Based on the difference between the frequency domain reconstructed image and the image to be detected, determine the frequency domain difference mask, and perform self-attention processing on the first image features through the frequency domain difference mask to obtain frequency domain suspected forgery features.

[0102] Here, the frequency domain difference mask refers to an image mask that partially obscures the first image features to control the processing area within those features. The frequency domain suspected forgery feature refers to the feature predicted from the frequency domain reconstructed image, used to identify forged regions in the image to be detected.

[0103] Specifically, the frequency domain processing branch may further include a first difference attention module, which generates a frequency domain difference mask and performs self-attention processing on the first image features based on the frequency domain difference mask to obtain suspected forgery features in the frequency domain. Here, self-attention processing refers to the process of highlighting the suspected forgery regions in the first image features using the frequency domain difference mask and removing irrelevant feature regions.

[0104] In one embodiment, a frequency domain difference mask is determined based on the difference between the frequency domain reconstructed image and the image to be detected, and self-attention processing is performed on the first image features using the frequency domain difference mask to obtain frequency domain suspected forgery features. This includes: determining the first image difference between the frequency domain reconstructed image and the image to be detected, and performing convolution and activation processing on the first image difference to obtain a frequency domain difference mask; applying the frequency domain difference mask to the first image features through a self-attention mechanism to obtain frequency domain suspected forgery features.

[0105] Among them, the attention mechanism mimics the internal process of biological observation behavior, that is, a mechanism that aligns internal experience with external senses to increase the precision of observation of certain areas, while the self-attention mechanism is an improvement on the attention mechanism, which reduces the dependence on external information and is better at capturing the internal correlation of features.

[0106] Specifically, when the frequency domain reconstructed image is obtained, the first difference attention module can subtract the frequency domain reconstructed image from the image to be detected to obtain the first image difference between the frequency domain reconstructed image and the image to be detected. To obtain the element values ​​of each mask element in the frequency domain difference mask, the first difference attention module also needs to perform convolution and activation processing on the first image difference to obtain the corresponding frequency domain difference mask. Specifically, the first difference attention module can first perform convolution processing on the first image difference to obtain the corresponding convolution result, and then perform activation processing on the convolution result to obtain the frequency domain difference mask.

[0107] Furthermore, since the frequency domain reconstructed image can be considered a more realistic image output by the image authenticity detection model, and the image to be detected may contain forged regions, regions in the image to be detected that differ from the frequency domain reconstructed image may be forged regions. Therefore, the frequency domain difference mask determined based on the first image difference between the frequency domain reconstructed image and the image to be detected can increase the observation precision of target features used to identify forged regions in the first image features, while reducing the observation precision of non-target features. When the frequency domain difference mask is obtained, the first difference attention module can apply the frequency domain difference mask to the first image features through a self-attention mechanism to obtain frequency domain suspected forged features. For example, the first difference attention module can recalculate the element values ​​of each first feature element in the first image features using the element values ​​of each mask element in the frequency domain difference mask, thereby applying the frequency domain difference mask to the first image features through a self-attention mechanism.

[0108] In one embodiment, the first difference attention module can determine the first pixel value of each pixel in the frequency domain reconstruction image and the second pixel value of each pixel in the image to be detected, and subtract the first pixel value from the second pixel value to obtain the first difference pixel value. The first image difference is obtained by combining the first difference pixel values.

[0109] In one embodiment, reference Figure 5 , Figure 5The diagram illustrates a scenario where the first difference attention module outputs frequency-domain suspected forgery features. While performing convolution and activation processing on the first image differences to obtain a frequency-domain difference mask, the first difference attention module can also perform convolution processing on the first image features to obtain first image convolutional features. The first difference attention module applies the frequency-domain difference mask to the first image convolutional features to obtain first image intermediate features, and adds the first image features to the first image intermediate features to obtain the final output frequency-domain suspected forgery features. Directly applying the frequency-domain difference mask to the first image convolutional features may lead to a performance degradation when the model has a high number of layers. Therefore, adding the first image features to the first image intermediate features yields frequency-domain suspected forgery features with more feature information, thereby improving model performance based on these more feature-rich features.

[0110] In this embodiment, frequency domain difference mask can be applied to the first image feature to obtain frequency domain suspected forgery features, thereby improving the extraction efficiency of frequency domain suspected forgery features.

[0111] Step S208: Based on the difference between the color reconstruction image and the image to be detected, determine the color difference mask, and perform self-attention processing on the second image features through the color difference mask to obtain color suspected forgery features.

[0112] Here, color difference mask refers to an image mask that partially obscures the second image features to control the processing area within those features. Color suspected forgery features refer to features predicted from the color reconstruction map used to identify forged areas in the image to be detected.

[0113] Specifically, the color processing branch may also include a second difference attention module, which generates a color difference mask and performs self-attention processing on the second image features based on the color difference mask to obtain color suspected forgery features. Here, self-attention processing refers to the process of highlighting the suspected forgery regions in the second image features using the color difference mask and removing irrelevant feature regions.

[0114] In one embodiment, a color difference mask is determined based on the difference between the color reconstructed image and the image to be detected, and self-attention processing is performed on the second image features using the color difference mask to obtain color suspected forgery features. This includes: determining the second image difference between the color reconstructed image and the image to be detected, and performing convolution and activation processing on the second image difference to obtain a color difference mask; applying the color difference mask to the second image features through a self-attention mechanism to obtain color suspected forgery features.

[0115] Specifically, when the color reconstruction map is obtained, the second difference attention module can subtract the color reconstruction map from the image to be detected to obtain the second image difference between the color reconstruction map and the image to be detected. To obtain the element values ​​of each mask element in the color difference mask, the second difference attention module also needs to perform convolution and activation processing on the second image difference to obtain the corresponding color difference mask. Specifically, the second difference attention module can first perform convolution processing on the second image difference to obtain the corresponding convolution result, and then perform activation processing on the convolution result to obtain the frequency domain difference mask.

[0116] Furthermore, since the color reconstruction map can be considered a more realistic image output by the image authenticity detection model, and the image to be detected may contain forged regions, regions in the image to be detected that differ from the color reconstruction map may be forged regions. Therefore, the color difference mask determined based on the second image difference between the color reconstruction map and the image to be detected can increase the observation precision of target features used to identify forged regions in the second image features, while reducing the observation precision of non-target features. When the color difference mask is obtained, the second difference attention module can apply the color difference mask to the second image features through a self-attention mechanism to obtain color-suspected forged features. For example, the second difference attention module can recalculate the element values ​​of each second feature element in the second image features using the element values ​​of each mask element in the color difference mask, thereby applying the color difference mask to the second image features through a self-attention mechanism.

[0117] In one embodiment, the second difference attention module can also determine the third pixel value of each pixel in the color reconstruction image and the second pixel value of each pixel in the image to be detected, and subtract the third pixel value from the second pixel value to obtain the second difference pixel value. By combining the second difference pixel values, the second image difference is obtained.

[0118] In the above embodiments, since color suspected forgery features can be obtained simply by applying the color difference mask to the second image features, the determination efficiency of color suspected forgery features is improved.

[0119] Step S210: Combine the suspected forgery features in the frequency domain and the suspected forgery features in the color domain to determine the authenticity detection result of the image to be detected.

[0120] Specifically, since frequency domain suspected forgery features reflect regions in the image to be detected that may be forged in the frequency domain, and color suspected forgery features reflect regions in the image to be detected that may be forged in the color space, the image authenticity detection model can combine these two features to obtain a probability value that the image to be detected is a genuine image, and determine the authenticity detection result based on this probability value. For example, when the probability value of being a genuine image is higher than a preset probability threshold, the image to be detected is determined to be a genuine image. By combining frequency domain suspected forgery features and color suspected forgery features to determine the authenticity detection result of the image to be detected, the accuracy of the authenticity detection result can be improved.

[0121] In one embodiment, when a forged region is determined to exist in the image to be detected based on frequency domain suspected forgery features, the image authenticity detection model can determine the location information of the forged region in the image to be detected, denoted as first location information. Similarly, when a forged region is determined to exist in the image to be detected based on color suspected forgery features, the image authenticity detection model can determine the location information of the forged region in the image to be detected, denoted as second location information. When the first location information matches the second location information—for example, when the first location information and the second location information are the same—it can be considered that the image region is determined to be a forged region both in the frequency domain and in the color space. Therefore, the image authenticity detection model determines that a forged image region does indeed exist in the image to be detected and identifies the image to be detected as a forged image. Furthermore, when the first location information and the second location information partially overlap, it can also be considered that the image authenticity detection model determines that a forged image region does indeed exist in the image to be detected and identifies the image to be detected as a forged image.

[0122] In the aforementioned image authenticity detection method, by acquiring the image to be detected, some information in the image can be filtered to obtain a frequency domain missing map and a color missing map. By generating the frequency domain missing map, image reconstruction can be performed based on the first image features in the frequency domain missing map to obtain a frequency domain reconstructed map; similarly, by generating the color missing map, image reconstruction can be performed based on the second image features in the color missing map to obtain a color reconstructed map. By generating the frequency domain reconstructed map, a frequency domain difference mask can be obtained, and thus, frequency domain suspected forgery features in the frequency domain space can be obtained based on the frequency domain difference mask; similarly, by generating the color reconstructed map, a color difference mask can be obtained, and thus, color suspected forgery features in the color space can be obtained based on the color difference mask. Therefore, by combining the frequency domain suspected forgery features in the frequency domain space and the color space suspected forgery features in the color space, the authenticity detection result can be output. Since the authenticity detection result is obtained by integrating frequency domain suspected forgery features in the frequency domain space and color suspected forgery features in the color space, compared with the traditional method of detecting certain specific forgery traces to detect the image to be tested, this application is not limited by specific forgery traces, thereby improving the generalization of the image authenticity detection model and thus improving the accuracy of the authenticity detection result.

[0123] In one embodiment, acquiring the image to be detected includes: acquiring the video to be detected, and extracting multiple video frames from the video to be detected according to the sampling frequency; using each extracted video frame as the image to be detected; the above image authenticity detection method further includes: combining the authenticity detection results corresponding to each image to be detected to determine the authenticity detection result of the video to be detected.

[0124] Specifically, computer equipment can perform authenticity checks on the video to be tested to determine whether there are any traces of forgery in the video. For example, to improve application security, before logging into the application or transferring resources through the application, an image acquisition device can capture the corresponding user's liveness verification video and determine whether there are forged faces in the liveness verification video. The final liveness verification result is determined based on the detected forged faces.

[0125] When the video to be detected is acquired, the computer device can extract multiple video frames from the video according to a preset sampling frequency, and use all extracted video frames as images to be detected. Furthermore, for each of the multiple images to be detected, the computer device can input the current image to be detected into an image authenticity detection model, and the model will output the authenticity detection result corresponding to the current image. The image authenticity detection model can detect each image sequentially, or it can perform parallel detection on multiple images; this embodiment does not limit this.

[0126] When the authenticity detection results for each image to be detected are obtained, the computer device can combine these results to obtain the authenticity detection result for the video to be detected. For example, the image authenticity detection model can output the probability value of each image to be detected being a real image. The computer device superimposes the probability values ​​of each image to be detected and then averages them to obtain the average probability value of each image to be detected being a real image. If the average probability value is higher than a preset probability threshold, the video to be detected is determined to be a real video; if the average probability value is lower than or equal to the preset probability threshold, the video to be detected is determined to be a fake video. When the video to be detected is a real video, it can be considered that the video content in the video to be detected has not undergone image editing processing; when the video to be detected is a fake video, it can be considered that at least part of the video content in the video to be detected has undergone image editing processing.

[0127] In one embodiment, the computer device may also sample from the video to be detected at equal intervals to obtain a preset number of video frames, for example, 50 frames are sampled from the video to be detected at equal intervals.

[0128] In one embodiment, when performing a face verification test on a video to determine whether a face in the video is a fake face, for each of the multiple video frames, the computer device identifies the face in the current video frame using a preset face recognition algorithm and selects the face region in the current video frame using a detection box. The computer device then expands the image region selected by the detection box by a preset factor, such as 1.2 times, based on the center of the face region, so that the final selected area includes the complete face and part of the background area. The computer device then crops the image region finally selected by the detection box to obtain the image to be detected. Here, a fake face refers to a face in the image that has been edited, such as by slimming the face or enlarging the eyes. Conversely, a real face refers to a face in the image that has not been edited.

[0129] In one embodiment, when the authenticity detection results corresponding to each image to be detected are obtained, the computer device can count the number of real images and the number of fake images in the images to be detected based on the authenticity detection results. If the number of real images is greater than the number of fake images, the video to be detected is determined to be a real video; if the number of real images is less than or equal to the number of fake images, the video to be detected is determined to be a fake video.

[0130] In the above embodiments, since image authenticity detection is performed on a preset number of extracted image frames, compared to performing image authenticity detection on every frame of the video to be detected, this embodiment can reduce the number of detections, thereby improving detection efficiency. Since the authenticity detection result of the video to be detected is determined by comprehensively considering the authenticity detection results corresponding to each image to be detected, the accuracy of the authenticity detection result of the video to be detected can also be improved.

[0131] In one embodiment, a frequency domain difference mask is applied to the first image features through a self-attention mechanism to obtain frequency domain suspected forgery features, including: determining the attention weight corresponding to each first feature element in the first image features according to the frequency domain difference mask; multiplying each first feature element by its corresponding attention weight to obtain frequency domain fusion features; and combining the frequency domain fusion features and the first image features to obtain frequency domain suspected forgery features.

[0132] Specifically, when the frequency domain difference mask is obtained, the first difference attention module can determine the attention weight corresponding to each first feature element in the first image feature based on the frequency domain difference mask. For example, the first difference attention module can use the element values ​​of the mask elements in the frequency domain difference mask as the attention weights of the first feature elements with the same positional information. Alternatively, the first difference attention module can also determine multiple mask elements with the same or adjacent positional information in the frequency domain difference mask based on the positional information of the current first feature element, and combine the element values ​​of the multiple mask elements to obtain the attention weight corresponding to the current first feature element.

[0133] Furthermore, the first difference attention module multiplies each first feature element by its corresponding attention weight, thereby recalculating the element values ​​of each first feature element in the first image feature using the element values ​​of each mask element in the frequency domain difference mask, to obtain the corresponding frequency domain fusion feature. To obtain features with richer information, the first difference attention module can also combine the frequency domain fusion feature and the first image feature to obtain a frequency domain suspected forgery feature. For example, the first difference attention module adds the frequency domain fusion feature to the first image feature to obtain the frequency domain suspected forgery feature.

[0134] In one embodiment, it can be achieved through formula H i，c (x)=M i，c (x)*T i，c (x) is used to obtain the frequency domain fusion features, where x is the input data, i is the spatial location, c is the channel index, and T is the frequency domain fusion feature. i，c (x) is the first feature element when the input is x, the spatial location is i, and the channel index is c, M i，c (x) is related to T i，c(x) corresponds to the attention weights. It's easy to understand that the size of the frequency domain difference mask can be the same as the size of the first image feature, M. i，c (x) can be the element value of the mask element in the frequency domain difference mask.

[0135] In the above embodiments, the attention weights corresponding to each first feature element can be determined by the frequency domain difference mask. This allows the feature regions suspected of being marked as forgeries in the first image features to be highlighted based on the attention weights, while removing irrelevant feature regions. This enables the corresponding authenticity detection results to be determined quickly and accurately based on the more prominent feature regions.

[0136] In one embodiment, a color difference mask is applied to the second image features through a self-attention mechanism to obtain color suspected forgery features, including: determining the attention weight corresponding to each second feature element in the second convolutional features according to the color difference mask; multiplying each second feature element by its corresponding attention weight to obtain color fusion features; and combining the color fusion features and the second image features to obtain color suspected forgery features.

[0137] Specifically, when the color difference mask is obtained, the second difference attention module can determine the attention weight corresponding to each second feature element in the second image features based on the color difference mask. For example, the second difference attention module can use the element values ​​of the mask elements in the frequency domain difference mask as the attention weights of the second feature elements with the same positional information. Further, the second difference attention module multiplies each second feature element by its corresponding attention weight to recalculate the element values ​​of each second feature element in the second image features using the element values ​​of each mask element in the color difference mask, thus obtaining the corresponding color fusion features. The second difference attention module combines the color fusion features and the second image features to obtain color suspected forgery features. For example, the second difference attention module adds the color fusion features to the second image features to obtain color suspected forgery features.

[0138] In one embodiment, when M i，c When the value of (y) is in the range of (0, 1], it can be obtained through formula H. i，c (y)=(1+M i，c (y))*T i，c (y) is used to obtain color suspected forgery features, where y is the input data, i is the spatial location, c is the channel index, and T is the channel index. i，c (y) is the second feature element when the input is y, the spatial location is i, and the channel index is c, M i，c (y) is related to T i，c (y) corresponds to the attention weight. It's easy to understand that M... i，c(y) can be the element value of the mask element in the color difference mask.

[0139] In the above embodiments, the attention weight corresponding to each second feature element can be determined by the color difference mask. This allows the feature regions in the second image features that are suspected to be marked as fake to be highlighted based on the attention weight, while removing irrelevant feature regions. This enables the corresponding authenticity detection results to be determined quickly and accurately based on more prominent features.

[0140] In one embodiment, determining the authenticity detection result of the image to be detected by combining frequency domain suspected forgery features and color suspected forgery features includes: performing feature supplementation processing on frequency domain suspected forgery features using color suspected forgery features to obtain corresponding frequency domain supplementary features; performing feature supplementation processing on color suspected forgery features using frequency domain suspected forgery features to obtain corresponding color supplementary features; and combining frequency domain supplementary features and color supplementary features to obtain the authenticity detection result of the image to be detected.

[0141] Specifically, to promote the fusion and complementarity of features extracted from the color space and frequency domain, the image authenticity detection model can also fuse information from both spaces for collaborative learning, thereby uncovering more comprehensive forgery information. When frequency domain suspected forgery features and color suspected forgery features are obtained, the image authenticity detection model can supplement the frequency domain suspected forgery features with the color suspected forgery features to obtain corresponding frequency domain supplementary features, and vice versa, to obtain corresponding color supplementary features. The model then combines the frequency domain supplementary features and the color supplementary features to obtain the authenticity detection result of the image to be detected. For example, the image authenticity detection model performs convolution processing on the frequency domain supplementary features and the color supplementary features respectively to obtain frequency domain supplementary convolutional features and color supplementary convolutional features. These features are then merged along the channel dimension to obtain merged features. The merged features are input into a global average pooling layer and a fully connected layer to obtain the final output authenticity detection result.

[0142] In one embodiment, the image authenticity detection model may include a collaborative fusion module. This module can perform feature supplementation processing on frequency domain suspected forgery features based on color suspected forgery features to obtain corresponding frequency domain supplemented features, and perform feature supplementation processing on color suspected forgery features based on frequency domain suspected forgery features to obtain corresponding color supplemented features. It is readily understood that the collaborative fusion module may include a frequency domain fusion structure and a color fusion structure. Through the frequency domain fusion structure, feature supplementation processing can be performed on frequency domain suspected forgery features to obtain frequency domain supplemented features; through the color fusion structure, feature supplementation processing can be performed on color suspected forgery features to obtain color supplemented features.

[0143] In one embodiment, reference Figure 6 , Figure 6 The diagram illustrates the output of the authenticity detection result in one embodiment. When frequency domain suspected forgery features and color suspected forgery features are obtained, the image authenticity detection model can perform multi-layer convolution processing on these features respectively. The convolutionally processed frequency domain suspected forgery features and color suspected forgery features are then input into a collaborative fusion module, which outputs frequency domain supplementary features and color supplementary features. Further, the image authenticity detection model performs convolution processing on the frequency domain supplementary features and color supplementary features respectively to obtain frequency domain supplementary convolutional features and color supplementary convolutional features. These features are then merged along the channel dimension to obtain merged features. The merged features are input into a global average pooling layer and a fully connected layer to obtain the final output authenticity detection result.

[0144] In the above embodiments, by performing feature supplementation and feature merging on the suspected forgery features in the frequency domain and the suspected forgery features in the color domain, the image authenticity detection model can fuse information from the two spaces for collaborative learning, thereby uncovering more comprehensive forgery information and obtaining more accurate authenticity detection results through more comprehensive forgery information.

[0145] In one embodiment, feature supplementation processing is performed on the frequency domain suspected forgery features using color suspected forgery features to obtain corresponding frequency domain supplementary features. This includes: determining a first vector sequence corresponding to the frequency domain suspected forgery features; performing convolution processing on the color forgery features to obtain color forgery convolution features; determining the position information of each first vector element in the first vector sequence and the position information of each third feature element in the color forgery convolution features; fusing the first vector elements and third feature elements with corresponding position information to obtain frequency domain vector fusion features; and merging the frequency domain vector fusion features and the frequency domain suspected forgery features to obtain frequency domain supplementary features.

[0146] Specifically, when frequency domain suspected forgery features and color suspected forgery features are obtained, the collaborative fusion module performs convolution and global average pooling on the frequency domain suspected forgery features to obtain a first vector sequence corresponding to the frequency domain suspected forgery features. Simultaneously, the collaborative fusion module performs convolution on the color suspected forgery features to obtain color forgery convolutional features. The collaborative fusion module determines the positional information of each first vector element in the first vector sequence and the positional information of each third feature element in the color forgery convolutional features. The positional information of the first vector elements may include their order in the first vector sequence. The positional information of the third feature elements may include their order in the color forgery convolutional features.

[0147] Furthermore, the collaborative fusion module fuses the first vector element and the third feature element, which have corresponding location information, to obtain the frequency domain vector fusion feature. For example, referencing Figure 7 The collaborative fusion module multiplies the first and third vector elements with the same location information to obtain frequency domain vector fusion features. To obtain features with richer information, the collaborative fusion module merges the frequency domain vector fusion features and frequency domain suspected forgery features to obtain supplementary frequency domain features. For example, ... Figure 7 As shown, the collaborative fusion module merges the frequency domain vector fusion features and the frequency domain suspected forgery features along the feature channel dimension to obtain the frequency domain supplementary features. Figure 7 A schematic diagram of a collaborative fusion module in one embodiment is shown.

[0148] In this embodiment, by fusing the first vector element and the third feature element, the information in the frequency domain space and the information in the color space can be integrated, thereby improving the efficiency of information integration.

[0149] In one embodiment, feature supplementation processing is performed on the color suspected forgery feature using frequency domain suspected forgery feature to obtain the corresponding color supplementation feature, including: determining a second vector sequence corresponding to the color suspected forgery feature; performing convolution processing on the frequency domain forgery feature to obtain a frequency domain forgery convolution feature; determining the position information of each second vector element in the second vector sequence and the position information of each fourth feature element in the frequency domain forgery convolution feature; fusing the second vector element and the fourth feature element with corresponding position information respectively to obtain a color vector fusion feature; and merging the color vector fusion feature and the color suspected forgery feature to obtain the color supplementation feature.

[0150] Specifically, when frequency domain suspected forgery features and color suspected forgery features are obtained, the collaborative fusion module performs convolution and global average pooling on the color suspected forgery features to obtain a second vector sequence corresponding to the color suspected forgery features. Simultaneously, the collaborative fusion module performs convolution on the frequency domain suspected forgery features to obtain frequency domain forgery convolution features. The collaborative fusion module determines the position information of each second vector element in the second vector sequence and the position information of each fourth feature element in the frequency domain forgery convolution features. The second vector elements and fourth feature elements with corresponding position information are fused to obtain a color vector fusion feature. The position information of the second vector elements may include their order in the second vector sequence. The position information of the fourth feature elements may include their order in the frequency domain forgery convolution features. For example, refer to... Figure 7The collaborative fusion module multiplies the second and fourth vector elements with the same positional information to obtain the color vector fusion feature. Further, the collaborative fusion module combines the color vector fusion feature and the color suspected forgery feature to obtain the color supplementary feature. For example, such as... Figure 7 As shown, the collaborative fusion module merges the color vector fusion features and the color suspected forgery features along the feature channel dimension to obtain the color supplementary features.

[0151] In this embodiment, by simply fusing the second vector element and the fourth feature element, the information in the frequency domain space and the information in the color space can be integrated, thereby improving the efficiency of information integration.

[0152] In one embodiment, the image authenticity detection method is executed by an image authenticity detection model, which is trained through a model training step. The model training step includes: acquiring a set of sample images and the corresponding sample label for each sample image in the set; obtaining a frequency domain prediction missing map based on partial frequency domain information from the sample images, and obtaining a color prediction missing map based on partial color information from the sample images; generating a frequency domain prediction reconstruction map based on a first prediction feature in the frequency domain prediction missing map, and generating a color prediction reconstruction map based on a second prediction feature in the color prediction missing map; and determining a frequency domain prediction mask based on the difference between the frequency domain prediction reconstruction map and the sample images. The first predicted feature is processed by self-attention using a frequency domain prediction mask to obtain a frequency domain prediction forgery feature. A color prediction mask is determined based on the difference between the color prediction reconstruction image and the sample image, and a second predicted feature is processed by self-attention using this mask to obtain a color prediction forgery feature. The frequency domain prediction forgery feature and the color prediction forgery feature are combined to obtain the authenticity prediction result of the sample image. The classification loss is determined based on the authenticity prediction result and the corresponding sample label. A target loss function is determined based on the classification loss, and the image authenticity detection model is trained using the target loss function until the training stops when the stopping condition is met, resulting in a trained image authenticity detection model.

[0153] Specifically, before performing authenticity detection based on the image authenticity detection model, the model can be trained by constructing a target loss function to adjust the model parameters and obtain a trained image authenticity detection model.

[0154] In one embodiment, the training steps of the image authenticity detection model further include: when the training sample is a real image, constructing a first reconstruction loss based on the difference between the training sample and the corresponding frequency domain predicted reconstruction map, and constructing a second reconstruction loss based on the difference between the training sample and the corresponding color predicted reconstruction map; and determining the target loss function through the first reconstruction loss, the second reconstruction loss, and the classification loss.

[0155] Specifically, in order for the image authenticity detection model to learn the essential information of real images and to reconstruct real samples better using the learned essential information of real images, so as to achieve the goal of including areas with forgery traces in areas that the image authenticity detection model cannot reconstruct well, the computer can also construct a first reconstruction loss and a second reconstruction loss, and improve the model's ability to reconstruct real images through reconstruction loss.

[0156] In one embodiment, reference Figure 8 , Figure 8 A schematic diagram of the overall framework of an image authenticity detection model in one embodiment is shown. Figure 8 As shown, the image authenticity detection model includes a frequency domain processing branch, a color processing branch, and a collaborative fusion module. The frequency domain processing branch includes a first difference attention module, and the color processing branch includes a second difference attention module. The frequency domain processing branch generates a missing frequency domain map and uses the first image features from the missing frequency domain map to generate a reconstructed frequency domain map. The color processing branch generates a missing color map and uses the second image features from the missing color map to generate a reconstructed color map. The first difference attention module generates a frequency domain difference mask and obtains suspected forgery features based on it. The second difference attention module generates a color difference mask and obtains suspected forgery features based on it. The collaborative fusion module performs feature supplementation processing on the suspected forgery features in the frequency domain to obtain supplemented frequency domain features, and on the suspected forgery features in the color domain to obtain supplemented color features. Therefore, the image authenticity detection model can obtain the authenticity detection result of the image to be detected based on the supplemented frequency domain features and the supplemented color features.

[0157] In one embodiment, such as Figure 9 As shown, a method for training an image authenticity detection model is provided. This method is illustrated by its application to a computer device, which can specifically be... Figure 1 The terminal or server in the system. The training method for this image authenticity detection model includes the following steps:

[0158] S902, obtain the sample image set and the sample label corresponding to each sample image in the sample image set.

[0159] Specifically, before using an image authenticity detection model to detect the authenticity of an image, the model needs to be trained. Computer devices can acquire a large number of sample images and their corresponding labels, allowing them to iteratively train the image authenticity detection model based on these images and labels.

[0160] S904, obtain a frequency domain prediction missing map based on partial frequency domain information in the sample image, and obtain a color prediction missing map based on partial color information in the sample image.

[0161] S906, Generate a frequency domain prediction reconstruction map based on the first prediction feature in the missing map of frequency domain prediction, and generate a color prediction reconstruction map based on the second prediction feature in the missing map of color prediction.

[0162] S908. Based on the difference between the frequency domain predicted reconstruction map and the sample image, determine the frequency domain prediction mask, and perform self-attention processing on the first predicted feature through the frequency domain prediction mask to obtain the frequency domain predicted fake feature.

[0163] S910: Based on the difference between the color prediction reconstruction image and the sample image, determine the color prediction mask, and perform self-attention processing on the second prediction feature through the color prediction mask to obtain the color prediction fake feature.

[0164] S912 combines frequency domain prediction forgery features and color prediction forgery features to obtain the authenticity prediction results of the sample images, and determines the classification loss based on the authenticity prediction results and the corresponding sample labels.

[0165] Specifically, when a sample image is acquired, the computer device can input the sample image into the image authenticity detection model to be trained. The model then outputs a authenticity prediction result for the sample image. The specific process of the image authenticity detection model outputting the prediction result can be referenced in the process described above for detecting the authenticity of the image to obtain the detection result. Since forgery traces in an image generally exist in the high-frequency components of the frequency domain, by filtering some high-frequency information and constraining the model to reconstruct the original frequency domain based on the filtered information, the model can learn how to better recover the frequency domain. This means learning the frequency domain distribution of a real face, and subsequently outputting a more accurate authenticity detection result based on the learning results.

[0166] Furthermore, since the sample labels may include classification labels, such as those containing the words "real image" and "fake image", when the authenticity prediction results are obtained, the computer device can also determine the corresponding classification loss based on the difference between the authenticity prediction results and the corresponding classification labels.

[0167] S914, determine the target loss function based on the classification loss, and train the image authenticity detection model using the target loss function until the training stops when the stopping condition is met, thus obtaining the trained image authenticity detection model; wherein, the trained image authenticity detection model is used to perform image authenticity detection on the image to be detected, and obtain the corresponding authenticity detection results.

[0168] Specifically, the computer device can determine the corresponding target loss function based on the classification loss, and train the image authenticity detection model using the target loss function to adjust the model parameters until a training stopping condition is met, resulting in a trained image authenticity detection model. The training stopping condition can be freely set according to needs; for example, it can be determined that the training stopping condition is met after a preset number of iterations. Once the trained image authenticity detection model is obtained, the computer device can use this model to perform image authenticity detection on the image to be detected.

[0169] In the above-described image authenticity detection model training method, by acquiring sample images, frequency domain prediction missing maps and color prediction missing maps corresponding to the sample images can be generated. Based on the frequency domain prediction missing maps, corresponding frequency domain prediction reconstruction maps are generated, and based on the color prediction missing maps, corresponding color prediction reconstruction maps are generated. By generating the frequency domain prediction reconstruction maps, frequency domain prediction forgery features can be obtained based on the differences between the frequency domain prediction reconstruction maps and the corresponding sample images; by generating the color prediction reconstruction maps, color prediction forgery features can be obtained based on the differences between the color prediction reconstruction maps and the corresponding sample images. Thus, by combining the frequency domain prediction forgery features in the frequency domain space and the color prediction forgery features in the color space, a authenticity prediction result can be obtained. Based on the authenticity prediction results, the difference between the authenticity prediction results and the corresponding sample labels can be determined, and the image authenticity detection model can be trained based on the target loss function to obtain a trained image authenticity detection model. Since the image authenticity detection model is trained by combining frequency domain prediction forgery features and color prediction forgery features, the trained image authenticity detection model can output more accurate frequency domain suspected forgery features and color suspected forgery features, thereby making the authenticity detection results obtained based on more accurate frequency domain suspected forgery features and color suspected forgery features more accurate.

[0170] In one embodiment, obtaining a sample image set includes: obtaining multiple real videos and at least one fake video corresponding to each real video; for each of the multiple real videos, selecting a target fake video from at least one fake video corresponding to the current real video; and extracting video frames from each real video and each target fake video to obtain a sample image set.

[0171] Specifically, the computer device can acquire multiple real videos, as well as multiple forged videos corresponding to each real video. To improve the balance of positive and negative sample images in the sample image set, for each of the multiple real videos, the computer device selects a target forged video from the multiple forged videos corresponding to the current real video, and extracts multiple video frames from the real video and the target forged video respectively according to a preset acquisition frequency to obtain a sample image set.

[0172] In one embodiment, reference Figure 10 When a real video is obtained, the computer device can use various image editing applications to edit the video content of the real video to obtain multiple fake videos. For example, the computer device can use a first image editing application to replace faces in the real video to obtain a first fake video corresponding to the real video, and use a second image editing application to smooth and slim faces in the real video to obtain a second fake video corresponding to the real video. The computer device can then filter out the target fake video from the first and second fake videos, and extract video frames from both the real video and the target fake video to obtain a sample image set. Figure 10 A schematic diagram of a sample image set in one embodiment is shown.

[0173] In one embodiment, when a sample image is acquired, the computer device can further perform data augmentation processing on the sample image to improve the generalization of the image authenticity detection model based on the data-augmented sample image. For example, the computer device can perform random horizontal flipping, random modeling, or random compression on the sample image to perform data augmentation processing.

[0174] In the above embodiments, by filtering out the target fake video from multiple fake videos, the number of positive and negative samples in the sample image set obtained based on the real video and the target fake video can be roughly the same, thereby making the image authenticity detection model trained by the sample image set with balanced positive and negative samples more accurate.

[0175] In one embodiment, before determining the target loss function based on the classification loss, the above image authenticity detection model training method further includes: when the training sample is a real image, constructing a first reconstruction loss based on the difference between the training sample and the corresponding frequency domain predicted reconstruction map, and constructing a second reconstruction loss based on the difference between the training sample and the corresponding color predicted reconstruction map; determining the target loss function based on the classification loss includes: determining the target loss function through the first reconstruction loss, the second reconstruction loss and the classification loss.

[0176] Specifically, in order for the image authenticity detection model to learn the essential information of real images and to reconstruct real samples better using the learned essential information of real images, so as to achieve the goal of including areas with forgery traces in areas that the image authenticity detection model cannot reconstruct well, the computer can also construct a reconstruction loss to improve the model's ability to reconstruct real images.

[0177] When the training samples are real images, the computer device can determine the difference between the training samples and the corresponding frequency domain predicted reconstruction map, and construct a first reconstruction loss based on the difference. When the training samples are real images, the computer device can also determine the difference between the training samples and the corresponding color predicted reconstruction map, and construct a second reconstruction loss based on the difference. The computer device determines the corresponding target loss function using the first reconstruction loss, the second reconstruction loss, and the classification loss, and trains the image authenticity detection model using the target loss function.

[0178] In one embodiment, the computer device can determine the first reconstruction loss using the following formula:

[0179]

[0180] Where R represents the real images in the sample image set. For frequency domain prediction and reconstruction, x i These are the training samples. It's easy to understand that the computer device can also determine the second reconstruction loss using this formula. When determining the second reconstruction loss based on this formula... This is the color prediction and reconstruction image.

[0181] In the above embodiments, by constructing a reconstruction loss, the image authenticity detection model can learn the essential information of the real image. As a result, when the image to be detected is a real image, the frequency domain reconstruction map and color reconstruction map reconstructed based on the learned essential information can be more accurate. When the image to be detected is a fake image, the frequency domain reconstruction map and color reconstruction map reconstructed based on the learned essential information can reflect the fake area.

[0182] In one embodiment, the above-mentioned image authenticity detection model training method further includes: determining a first training sample pair and a second training sample pair in the training sample set; the first training sample pair includes two training samples with the same authenticity category; the second training sample pair includes two training samples with different authenticity categories; determining a first image distance between the first predicted features corresponding to the two training samples in the first training sample pair, and a second image distance between the second predicted features corresponding to the two training samples in the first training sample pair; determining a third image distance between the first predicted features corresponding to the two training samples in the second training sample pair, and a fourth image distance between the second predicted features corresponding to the two training samples in the second training sample pair; determining a first constraint loss based on the first image distance and the third image distance; determining a second constraint loss based on the second image distance and the fourth image distance; and determining a target loss function through a first reconstruction loss, a second reconstruction loss, and a classification loss, including: determining the target loss function through the first constraint loss, the second constraint loss, the first reconstruction loss, the second reconstruction loss, and the classification loss.

[0183] Specifically, to improve the image authenticity detection model's ability to distinguish between genuine and fake images, the computer device can also set a constraint loss. This constraint loss improves the accuracy of the first and second image features output by the image authenticity detection model. The computer device determines a first training sample pair and a second training sample pair in the training sample set. The first training sample pair includes two training samples with the same authenticity category; the second training sample pair includes two training samples with different authenticity categories. For example, both training samples in the first training sample pair are genuine images, and one training sample in the second training sample pair is a genuine image while the other is a fake image. For the first training sample pair, the computer device determines the first predicted feature corresponding to each training sample in the first training sample pair and determines the first image distance between the corresponding first predicted features of each training sample. Correspondingly, the computer device determines the second predicted feature corresponding to each training sample in the first training sample pair and determines the second image distance between the corresponding second predicted features of each training sample.

[0184] Furthermore, for the second training sample pair, the computer device determines the first predicted feature corresponding to each training sample in the second training sample pair, and determines the third image distance between the first predicted features corresponding to each training sample. Correspondingly, the computer device determines the second predicted feature corresponding to each training sample in the second training sample pair, and determines the fourth image distance between the second predicted features corresponding to each training sample. Wherein, the first image distance, second image distance, third image distance, and fourth image distance can all be cosine distances.

[0185] Furthermore, the computer device determines a first constraint loss based on the first image distance and the third image distance; it determines a second constraint loss based on the second image distance and the fourth image distance; and it determines a target loss function based on the first constraint loss, the second constraint loss, the first reconstruction loss, the second reconstruction loss, and the classification loss. The image authenticity detection model is then trained using this target loss function. For example, the computer device can determine the weights corresponding to the first constraint loss, the second constraint loss, the first reconstruction loss, the second reconstruction loss, and the classification loss, and then perform a weighted summation of these weights to obtain the target loss function.

[0186] In one embodiment, the computer device can determine the first constraint loss using the following formula:

[0187]

[0188]

[0189] Where S is the first training sample pair, D is the second training sample pair, and F... * This is the first predicted feature corresponding to the *training sample. It is easily understood that the computer device can also determine the second constraint loss using the above formula. When the second constraint loss is determined based on the above formula, F... * This is the second predicted feature corresponding to the *training sample.

[0190] In the above embodiments, by generating constraint loss, the image authenticity detection model trained based on constraint loss can output more accurate image features.

[0191] This application also provides an application scenario in which the above-described image authenticity detection method is applied. Specifically, the image authenticity detection method is applied in this scenario as follows:

[0192] On some multimedia platforms, users can freely upload edited, face-swapped videos. The widespread dissemination of these videos has led to a decline in media credibility and can easily mislead users. When a user uploads a video to a multimedia platform, the platform can use an image authenticity detection model to screen the received video, selecting multiple image frames to obtain multiple images to be detected. Each image is then subjected to authenticity testing to obtain its corresponding detection result. If a predetermined number of fake images are found among the detected images, the video is identified as a location-based video, and a prominent label, such as "made by Deepfakes," is added to the detected fake video. This ensures the credibility of the video content and guarantees public trust.

[0193] This application also provides another application scenario in which the above-described image authenticity detection method is applied. Specifically, the image authenticity detection method is applied in this scenario as follows:

[0194] Before freezing a user account through an account management application, to ensure account security, the application can capture the user's facial image using an image acquisition device and send it to a server for facial verification. Upon receiving the user's facial image, the server inputs it into an image authenticity detection model. The model then outputs the authenticity detection result. If the user's facial image is fake, the account management application will refuse to freeze the account; if the user's facial image is genuine, the application will freeze the account.

[0195] The above application scenarios are merely illustrative. It is understood that the application of the image authenticity detection methods provided in the embodiments of this application is not limited to the above scenarios. For example, they can also be applied to scenarios such as face verification and judicial verification.

[0196] In one specific embodiment, reference Figure 11 This paper provides a method for detecting the authenticity of images, which includes the following steps:

[0197] S1102, obtain the sample image set and the sample label corresponding to each sample image in the sample image set; obtain the frequency domain prediction missing map based on some frequency domain information in the sample images, and obtain the color prediction missing map based on some color information in the sample images; generate the frequency domain prediction reconstruction map based on the first prediction feature in the frequency domain prediction missing map, and generate the color prediction reconstruction map based on the second prediction feature in the color prediction missing map.

[0198] S1104. Based on the difference between the frequency domain predicted reconstruction map and the sample image, determine the frequency domain prediction mask, and perform self-attention processing on the first prediction feature through the frequency domain prediction mask to obtain the frequency domain prediction fake feature; based on the difference between the color predicted reconstruction map and the sample image, determine the color prediction mask, and perform self-attention processing on the second prediction feature through the color prediction mask to obtain the color prediction fake feature.

[0199] S1106, combining frequency domain prediction forgery features and color prediction forgery features, obtain the true or false prediction results of the sample image, and determine the classification loss based on the true or false prediction results and the corresponding sample labels; when the training sample is a real image, construct the first reconstruction loss based on the difference between the training sample and the corresponding frequency domain prediction reconstruction image, and construct the second reconstruction loss based on the difference between the training sample and the corresponding color prediction reconstruction image.

[0200] S1108: The image authenticity detection model is trained using the target loss function until the training stops when the stopping condition is met, thus obtaining the trained image authenticity detection model.

[0201] S1110: Acquire the video to be detected, and extract multiple video frames from the video to be detected according to the sampling frequency, for example, sample 50 frames at equal intervals from the video to be detected; use each extracted video frame as the image to be detected.

[0202] S1112, the image to be detected is transformed from the image spatial domain to the frequency domain to obtain a frequency domain image. For example, a discrete cosine transform is performed on the image to be detected to obtain a frequency domain image. Frequency domain information with spatial frequencies greater than a preset frequency threshold in the frequency domain image is filtered to obtain target frequency domain information in the image to be detected. For example, high-frequency information is filtered to obtain target frequency domain information. The target frequency domain information is transformed from the frequency domain to the image spatial domain. For example, an inverse discrete cosine transform is performed on the target frequency domain information to obtain a frequency domain missing map.

[0203] S1114, the image to be detected is segmented to obtain an image grid containing multiple image slices; at least one image slice in the image grid is converted into a grayscale image to obtain the corresponding color missing image.

[0204] S1116, Generate a frequency domain reconstructed map based on the first image features in the frequency domain missing map, for example, by encoding the frequency domain missing map to obtain the first image features, and by decoding the first image features to obtain the frequency domain reconstructed map; and Generate a color reconstructed map based on the second image features in the color missing map, for example, by encoding the color missing map to obtain the second image features, and by decoding the second image features to obtain the color reconstructed map.

[0205] S1118, determine the first image difference between the frequency domain reconstructed image and the image to be detected. For example, subtract the frequency domain reconstructed image from the image to be detected to obtain the first image difference, and perform convolution and activation processing on the first image difference to obtain the frequency domain difference mask.

[0206] S1120, Based on the frequency domain difference mask, determine the attention weight corresponding to each first feature element in the first image features. For example, use the element value of the mask element in the frequency domain difference mask as the attention weight of the first feature element at the corresponding position. Multiply each first feature element by its corresponding attention weight to obtain the frequency domain fusion feature. Combine the frequency domain fusion feature and the first image features to obtain the frequency domain suspected forgery feature. For example, add the frequency domain fusion feature and the first image features to obtain the frequency domain suspected forgery feature.

[0207] S1122, determine the second image difference between the color reconstruction map and the image to be detected, and perform convolution and activation processing on the second image difference to obtain the color difference mask.

[0208] S1124. Based on the color difference mask, determine the attention weight corresponding to each second feature element in the second convolution feature; multiply each second feature element by its corresponding attention weight to obtain the color fusion feature; combine the color fusion feature and the second image feature to obtain the color suspected forgery feature.

[0209] S1126, determine the first vector sequence corresponding to the suspected forgery features in the frequency domain. For example, perform convolution and global average pooling on the suspected forgery features in the frequency domain to obtain the first vector sequence; perform convolution on the color forgery features to obtain the color forgery convolution features.

[0210] S1128, determine the position information of each first vector element in the first vector sequence and the position information of each third feature element in the color spoofing convolution feature; fuse the first vector elements and third feature elements with corresponding position information respectively to obtain the frequency domain vector fusion feature; merge the frequency domain vector fusion feature and the frequency domain suspected spoofing feature to obtain the frequency domain supplementary feature, for example, merge the frequency domain vector fusion feature and the frequency domain suspected spoofing feature along the channel dimension to obtain the frequency domain supplementary feature.

[0211] S1130, determine the second vector sequence corresponding to the suspected color forgery feature; perform convolution processing on the frequency domain forgery feature to obtain the frequency domain forgery convolution feature.

[0212] S1132, determine the position information of each second vector element in the second vector sequence and the position information of each fourth feature element in the frequency domain fake convolution feature; fuse the second vector elements and fourth feature elements with corresponding position information respectively to obtain the color vector fusion feature; merge the color vector fusion feature and the color suspected fake feature to obtain the color supplement feature.

[0213] S1134, combine frequency domain supplementary features and color supplementary features to obtain the authenticity detection result of the image to be detected. For example, merge the frequency domain supplementary features and color supplementary features to obtain merged features, and obtain the authenticity detection result based on the merged features; combine the authenticity detection results corresponding to each image to be detected to determine the authenticity detection result of the video to be detected.

[0214] The aforementioned image authenticity detection method acquires the image to be detected and filters out some information from it, obtaining a frequency domain missing map and a color missing map. By generating the frequency domain missing map, image reconstruction can be performed based on the first image features in the frequency domain missing map, resulting in a frequency domain reconstructed map. Similarly, by generating the color missing map, image reconstruction can be performed based on the second image features in the color missing map, resulting in a color reconstructed map. By generating the frequency domain reconstructed map, a frequency domain difference mask can be obtained, which in turn allows for the extraction of suspected frequency domain forgery features in the frequency domain space. Furthermore, by generating the color reconstructed map, a color difference mask can be obtained, which in turn allows for the extraction of suspected color forgery features in the color space. Thus, by combining the suspected frequency domain forgery features and the suspected color forgery features in the color space, the authenticity detection result can be output. Since the authenticity detection result is obtained by integrating frequency domain suspected forgery features in the frequency domain space and color suspected forgery features in the color space, compared with the traditional method of detecting certain specific forgery traces to detect the image to be tested, this application is not limited by specific forgery traces, thereby improving the generalization of the image authenticity detection model and thus improving the accuracy of the authenticity detection result.

[0215] In one specific embodiment, reference Figure 12 A training method for an image authenticity detection model is provided, including the following steps:

[0216] S1202, acquire multiple real videos and at least one fake video corresponding to each real video; for each of the multiple real videos, select a target fake video from at least one fake video corresponding to the current real video; extract video frames from each real video and each target fake video respectively, for example, sample 50 frames at equal intervals from the real video and the target fake video respectively to obtain a sample image set, and obtain the sample label corresponding to each sample image in the sample image set.

[0217] S1204, obtain a frequency domain prediction missing map based on partial frequency domain information in the sample image, and obtain a color prediction missing map based on partial color information in the sample image.

[0218] S1206, Generate a frequency domain prediction reconstruction map based on the first prediction feature in the missing frequency domain prediction map, and generate a color prediction reconstruction map based on the second prediction feature in the missing color prediction map.

[0219] S1208. Based on the difference between the frequency domain predicted reconstruction map and the sample image, determine the frequency domain prediction mask, and perform self-attention processing on the first predicted feature through the frequency domain prediction mask to obtain the frequency domain predicted fake feature.

[0220] S1210: Based on the difference between the color prediction reconstruction image and the sample image, determine the color prediction mask, and perform self-attention processing on the second prediction feature through the color prediction mask to obtain the color prediction fake feature.

[0221] S1212 combines frequency domain prediction forgery features and color prediction forgery features to obtain the authenticity prediction results of the sample image, and determines the classification loss based on the authenticity prediction results and the corresponding sample labels. For example, the classification loss is determined based on the difference between the authenticity prediction results and the classification labels.

[0222] S1214, when the training sample is a real image, construct a first reconstruction loss based on the difference between the training sample and the corresponding frequency domain predicted reconstruction map, for example, by subtracting the training sample from the frequency domain predicted reconstruction map to obtain the first reconstruction loss, and construct a second reconstruction loss based on the difference between the training sample and the corresponding color predicted reconstruction map, for example, by subtracting the training sample from the color predicted reconstruction map to obtain the second reconstruction loss.

[0223] S1216, determine the first training sample pair and the second training sample pair in the training sample set; the first training sample pair includes two training samples with the same true / false category; the second training sample pair includes two training samples with different true / false categories; for example, both training samples in the first training sample pair are real images, and one training sample in the second training sample pair is a real image and the other is a fake image.

[0224] S1218, determine the first image distance between the first predicted features corresponding to the two training samples in the first training sample pair, and the second image distance between the second predicted features corresponding to the two training samples; determine the third image distance between the first predicted features corresponding to the two training samples in the second training sample pair, and the fourth image distance between the second predicted features corresponding to the two training samples, wherein the first image distance, the second image distance, the third image distance and the fourth image distance can all be cosine distances.

[0225] S1220, determine a first constraint loss based on the first image distance and the third image distance, for example, based on the difference between the first image distance and the third image distance; determine a second constraint loss based on the second image distance and the fourth image distance, based on the difference between the second image distance and the fourth image distance.

[0226] S1222, determine the target loss function by using the first constraint loss, the second constraint loss, the first reconstruction loss, the second reconstruction loss, and the classification loss. For example, the target loss function can be obtained by weighted summation of the first constraint loss, the second constraint loss, the first reconstruction loss, the second reconstruction loss, and the classification loss.

[0227] S1224, The image authenticity detection model is trained using the target loss function until the training stops when the stopping condition is met, resulting in a trained image authenticity detection model; the trained image authenticity detection model is used to detect the authenticity of the image to be detected and obtain the corresponding authenticity detection results.

[0228] The aforementioned image authenticity detection model training method, by acquiring sample images, generates corresponding frequency domain prediction missing maps and color prediction missing maps. Based on the frequency domain prediction missing maps, it generates corresponding frequency domain prediction reconstruction maps, and based on the color prediction missing maps, it generates corresponding color prediction reconstruction maps. By generating the frequency domain prediction reconstruction maps, frequency domain prediction forgery features can be obtained based on the differences between the frequency domain prediction reconstruction maps and the corresponding sample images; similarly, by generating the color prediction reconstruction maps, color prediction forgery features can be obtained based on the differences between the color prediction reconstruction maps and the corresponding sample images. Thus, by combining the frequency domain prediction forgery features in the frequency domain space and the color prediction forgery features in the color space, a authenticity prediction result can be obtained. Based on the differences between the authenticity prediction results and the corresponding sample labels, a target loss function can be determined. Therefore, the image authenticity detection model can be trained based on the target loss function to obtain a trained image authenticity detection model. Since the image authenticity detection model is trained by combining frequency domain prediction forgery features and color prediction forgery features, the trained image authenticity detection model can output more accurate frequency domain suspected forgery features and color suspected forgery features, thereby making the authenticity detection results obtained based on more accurate frequency domain suspected forgery features and color suspected forgery features more accurate.

[0229] It should be understood that, although Figure 2 , Figure 9 , Figures 11-12 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 , Figure 9 , Figures 11-12 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0230] In one embodiment, such as Figure 13As shown, an image authenticity detection device 1300 is provided. This device can be a software module, a hardware module, or a combination of both as part of a computer device. Specifically, the device includes: a reconstructed image generation module 1302, a suspected feature determination module 1304, and a result output module 1306, wherein:

[0231] The image reconstruction generation module 1302 is used to acquire the image to be detected, and obtain a frequency domain missing map based on some frequency domain information in the image to be detected, and obtain a color missing map based on some color information in the image to be detected; generate a frequency domain reconstruction map based on the first image feature in the frequency domain missing map, and generate a color reconstruction map based on the second image feature in the color missing map.

[0232] The suspected feature determination module 1304 is used to determine a frequency domain difference mask based on the difference between the frequency domain reconstructed image and the image to be detected, and to perform self-attention processing on the first image features through the frequency domain difference mask to obtain frequency domain suspected forgery features; and to determine a color difference mask based on the difference between the color reconstructed image and the image to be detected, and to perform self-attention processing on the second image features through the color difference mask to obtain color suspected forgery features.

[0233] The result output module 1306 is used to combine frequency domain suspected forgery features and color suspected forgery features to determine the authenticity detection result of the image to be detected.

[0234] In one embodiment, the reconstructed image generation module 1302 is further configured to acquire the video to be detected and extract multiple video frames from the video to be detected according to the sampling frequency; each extracted video frame is used as an image to be detected. The image authenticity detection device is further configured to integrate the authenticity detection results corresponding to each image to be detected to determine the authenticity detection result of the video to be detected.

[0235] In one embodiment, the reconstructed image generation module 1302 further includes a missing map generation module 1321, which is used to convert the image to be detected from the image spatial domain to the frequency domain to obtain a frequency domain image; filter the frequency domain information in the frequency domain image whose spatial frequency is greater than a preset frequency threshold to obtain the target frequency domain information in the image to be detected; and convert the target frequency domain information from the frequency domain to the image spatial domain to obtain a frequency domain missing map.

[0236] In one embodiment, the missing image generation module 1321 is further configured to perform segmentation processing on the image to be detected to obtain an image grid including multiple image slices; and convert at least one image slice in the image grid into a grayscale image to obtain a corresponding color missing image.

[0237] In one embodiment, the suspected feature determination module 1304 further includes a frequency domain mask generation module 1341, which is used to determine the first image difference between the frequency domain reconstructed image and the image to be detected, and to perform convolution and activation processing on the first image difference to obtain a frequency domain difference mask; and to apply the frequency domain difference mask to the first image features through a self-attention mechanism to obtain frequency domain suspected forgery features.

[0238] In one embodiment, the frequency domain mask generation module 1341 is further configured to determine the attention weight corresponding to each first feature element in the first image feature according to the frequency domain difference mask; multiply each first feature element by its corresponding attention weight to obtain the frequency domain fusion feature; and combine the frequency domain fusion feature and the first image feature to obtain the frequency domain suspected forgery feature.

[0239] In one embodiment, the suspected feature determination module 1304 further includes a color mask generation module 1342, which is used to determine the second image difference between the color reconstruction map and the image to be detected, and to perform convolution and activation processing on the second image difference to obtain a color difference mask; and to apply the color difference mask to the second image features through a self-attention mechanism to obtain color suspected forgery features.

[0240] In one embodiment, the color mask generation module 1342 is further configured to determine the attention weight corresponding to each second feature element in the second convolutional feature based on the color difference mask; multiply each second feature element by its corresponding attention weight to obtain the color fusion feature; and combine the color fusion feature and the second image feature to obtain the color suspected forgery feature.

[0241] In one embodiment, the result output module 1306 is further configured to perform feature supplementation processing on the frequency domain suspected forgery features using color suspected forgery features to obtain corresponding frequency domain supplementary features; perform feature supplementation processing on the color suspected forgery features using frequency domain suspected forgery features to obtain corresponding color supplementary features; and combine the frequency domain supplementary features and the color supplementary features to obtain the authenticity detection result of the image to be detected.

[0242] In one embodiment, the result output module 1306 further includes a feature fusion module 1361, used to determine a first vector sequence corresponding to the suspected forgery feature in the frequency domain; perform convolution processing on the color forgery feature to obtain a color forgery convolution feature; determine the position information of each first vector element in the first vector sequence and the position information of each third feature element in the color forgery convolution feature; fuse the first vector element and the third feature element with corresponding position information respectively to obtain a frequency domain vector fusion feature; and merge the frequency domain vector fusion feature and the suspected forgery feature in the frequency domain to obtain a frequency domain supplementary feature.

[0243] In one embodiment, the feature fusion module 1361 is further configured to: determine a second vector sequence corresponding to the color suspected forgery feature; perform convolution processing on the frequency domain forgery feature to obtain a frequency domain forgery convolution feature; determine the position information of each second vector element in the second vector sequence and the position information of each fourth feature element in the frequency domain forgery convolution feature; fuse the second vector element and the fourth feature element with corresponding position information respectively to obtain a color vector fusion feature; and merge the color vector fusion feature and the color suspected forgery feature to obtain a color supplement feature.

[0244] In one embodiment, such as Figure 14 As shown, an image authenticity detection model training device 1400 is provided. This device can be a software module, a hardware module, or a combination of both as part of a computer device. Specifically, the device includes: a prediction reconstruction map generation module 1402, a prediction feature determination module 1404, and a training module 1406, wherein:

[0245] The prediction reconstruction map generation module 1402 is used to obtain a sample image set and the sample label corresponding to each sample image in the sample image set; obtain a frequency domain prediction missing map based on some frequency domain information in the sample images, and obtain a color prediction missing map based on some color information in the sample images; generate a frequency domain prediction reconstruction map based on the first prediction feature in the frequency domain prediction missing map, and generate a color prediction reconstruction map based on the second prediction feature in the color prediction missing map.

[0246] The prediction feature determination module 1404 is used to determine a frequency domain prediction mask based on the difference between the frequency domain prediction reconstruction map and the sample image, and to perform self-attention processing on the first prediction feature through the frequency domain prediction mask to obtain a frequency domain prediction fake feature; and to determine a color prediction mask based on the difference between the color prediction reconstruction map and the sample image, and to perform self-attention processing on the second prediction feature through the color prediction mask to obtain a color prediction fake feature.

[0247] Training module 1406 is used to integrate frequency domain prediction forgery features and color prediction forgery features to obtain the authenticity prediction results of sample images, and to determine the classification loss based on the authenticity prediction results and corresponding sample labels; the target loss function is determined based on the classification loss, and the image authenticity detection model is trained through the target loss function until the training stops when the stopping condition is met, resulting in a trained image authenticity detection model; the trained image authenticity detection model is used to perform image authenticity detection on the image to be detected, and obtain the corresponding authenticity detection results.

[0248] In one embodiment, the prediction and reconstruction map generation module 1402 is further configured to acquire multiple real videos and at least one fake video corresponding to each real video; for each of the multiple real videos, a target fake video is selected from at least one fake video corresponding to the current real video; and video frames are extracted from each real video and each target fake video to obtain a sample image set.

[0249] In one embodiment, the image authenticity detection model training device 1400 is further configured to construct a first reconstruction loss based on the difference between the training sample and the corresponding frequency domain predicted reconstruction map when the training sample is a real image, and to construct a second reconstruction loss based on the difference between the training sample and the corresponding color predicted reconstruction map; and to determine the target loss function through the first reconstruction loss, the second reconstruction loss and the classification loss.

[0250] In one embodiment, the image authenticity detection model training device 1400 is further configured to determine a first training sample pair and a second training sample pair in the training sample set; the first training sample pair includes two training samples with the same authenticity category; the second training sample pair includes two training samples with different authenticity categories; determine a first image distance between the first predicted features corresponding to the two training samples in the first training sample pair, and a second image distance between the second predicted features corresponding to the two training samples in the first training sample pair; determine a third image distance between the first predicted features corresponding to the two training samples in the second training sample pair, and a fourth image distance between the second predicted features corresponding to the two training samples in the second training sample pair; determine a first constraint loss based on the first image distance and the third image distance; determine a second constraint loss based on the second image distance and the fourth image distance; and determine a target loss function using the first constraint loss, the second constraint loss, the first reconstruction loss, the second reconstruction loss, and the classification loss.

[0251] Specific limitations regarding the image authenticity detection device and the image authenticity detection model training device can be found in the limitations regarding the image authenticity detection method and the image authenticity detection model training method described above, and will not be repeated here. Each module in the aforementioned image authenticity detection device and image authenticity detection model training device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0252] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 15As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores image authenticity detection data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an image authenticity detection method and an image authenticity detection model training method.

[0253] Those skilled in the art will understand that Figure 15 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0254] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0255] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0256] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0257] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0258] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0259] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for detecting the authenticity of an image, characterized in that, The method includes: The image to be detected is acquired, and a frequency domain missing map is obtained based on some frequency domain information in the image to be detected, and a color missing map is obtained based on some color information in the image to be detected. A frequency domain reconstruction map is generated based on the first image features in the frequency domain missing map, and a color reconstruction map is generated based on the second image features in the color missing map; Based on the difference between the frequency domain reconstructed image and the image to be detected, a frequency domain difference mask is determined, and the first image features are subjected to self-attention processing using the frequency domain difference mask to obtain frequency domain suspected forgery features. Based on the difference between the color reconstruction map and the image to be detected, a color difference mask is determined, and the second image features are subjected to self-attention processing using the color difference mask to obtain color suspected forgery features; By combining the frequency domain suspected forgery features and the color suspected forgery features, the authenticity detection result of the image to be detected is determined.

2. The method according to claim 1, characterized in that, The step of obtaining the frequency domain missing map based on partial frequency domain information in the image to be detected includes: The image to be detected is transformed from the image spatial domain to the frequency domain to obtain a frequency domain image; Filter out frequency domain information in the frequency domain image whose spatial frequency is greater than a preset frequency threshold to obtain the target frequency domain information in the image to be detected; The target frequency domain information is converted from the frequency domain to the image spatial domain to obtain a frequency domain missing map.

3. The method according to claim 1, characterized in that, The step of obtaining a color missing map based on partial color information in the image to be detected includes: The image to be detected is segmented to obtain an image grid comprising multiple image slices; At least one image slice in the image grid is converted into a grayscale image to obtain the corresponding color missing image.

4. The method according to claim 1, characterized in that, The step of determining a frequency domain difference mask based on the difference between the frequency domain reconstructed image and the image to be detected, and performing self-attention processing on the first image features using the frequency domain difference mask to obtain frequency domain suspected forgery features includes: A first image difference between the frequency domain reconstructed image and the image to be detected is determined, and the first image difference is convolved and activated to obtain a frequency domain difference mask. By applying the frequency domain difference mask to the first image features through a self-attention mechanism, frequency domain suspected forgery features are obtained.

5. The method according to claim 4, characterized in that, The step of applying the frequency domain difference mask to the first image features through a self-attention mechanism to obtain frequency domain suspected forgery features includes: Based on the frequency domain difference mask, determine the attention weight corresponding to each first feature element in the first image feature; Each first feature element is multiplied by its corresponding attention weight to obtain the frequency domain fusion feature; By combining the frequency domain fusion features and the first image features, frequency domain suspected forgery features are obtained.

6. The method according to claim 1, characterized in that, The step of determining a color difference mask based on the difference between the reconstructed color image and the image to be detected, and performing self-attention processing on the second image features using the color difference mask to obtain color suspected forgery features includes: The second image difference between the color reconstruction map and the image to be detected is determined, and the second image difference is convolved and activated to obtain a color difference mask; By using a self-attention mechanism, the color difference mask is applied to the second image feature to obtain color suspected forgery features.

7. The method according to claim 6, characterized in that, The step of applying the color difference mask to the second image features through a self-attention mechanism to obtain color suspected forgery features includes: Based on the color difference mask, determine the attention weight corresponding to each second feature element in the second image feature; Each second feature element is multiplied by its corresponding attention weight to obtain the color fusion feature; Combining the color fusion features and the second image features, we obtain the color suspected forgery features.

8. The method according to claim 1, characterized in that, The determination of the authenticity detection result of the image to be detected by combining the frequency domain suspected forgery features and the color suspected forgery features includes: The frequency domain suspected forgery features are supplemented by the color suspected forgery features to obtain the corresponding frequency domain supplemented features. The color suspected forgery features are supplemented by the frequency domain suspected forgery features to obtain the corresponding color supplementation features; By combining the frequency domain supplementary features and the color supplementary features, the authenticity detection result of the image to be detected is obtained.

9. The method according to claim 8, characterized in that, The step of performing feature supplementation processing on the frequency domain suspected forgery features using the color suspected forgery features to obtain corresponding frequency domain supplemented features includes: Determine the first vector sequence corresponding to the suspected forgery features in the frequency domain; The color forgery features are convolved to obtain color forgery convolutional features; Determine the position information of each first vector element in the first vector sequence, and the position information of each third feature element in the color spoofing convolutional feature; The first vector element and the third feature element, which have corresponding location information, are fused to obtain the frequency domain vector fusion feature. The frequency domain vector fusion feature and the frequency domain suspected forgery feature are combined to obtain the frequency domain supplementary feature.

10. The method according to claim 8, characterized in that, The step of performing feature supplementation processing on the color suspected forgery features using the frequency domain suspected forgery features to obtain corresponding color supplementation features includes: Determine the second vector sequence corresponding to the suspected color forgery features; The frequency domain forgery features are convolved to obtain frequency domain forgery convolution features; Determine the position information of each second vector element in the second vector sequence, and the position information of each fourth feature element in the frequency domain fake convolutional feature; The second vector element and the fourth feature element, which have corresponding position information, are fused separately to obtain the color vector fusion feature; The color vector fusion feature and the color suspected forgery feature are combined to obtain the color supplement feature.

11. The method according to any one of claims 1 to 10, characterized in that, The image authenticity detection method is executed by an image authenticity detection model, which is trained through a model training step, including: Obtain the sample image set and the sample label corresponding to each sample image in the sample image set; A frequency domain prediction missing map is obtained based on partial frequency domain information in the sample image, and a color prediction missing map is obtained based on partial color information in the sample image; A frequency domain prediction reconstruction map is generated based on the first prediction feature in the frequency domain prediction missing map, and a color prediction reconstruction map is generated based on the second prediction feature in the color prediction missing map. Based on the difference between the frequency domain prediction reconstruction map and the sample image, a frequency domain prediction mask is determined, and the first prediction feature is subjected to self-attention processing through the frequency domain prediction mask to obtain a frequency domain prediction forgery feature. Based on the difference between the color prediction reconstruction map and the sample image, a color prediction mask is determined, and the second prediction feature is subjected to self-attention processing through the color prediction mask to obtain the color prediction fake feature; By combining the frequency domain prediction forgery features and the color prediction forgery features, the authenticity prediction result of the sample image is obtained, and the classification loss is determined based on the authenticity prediction result and the corresponding sample label. The target loss function is determined based on the classification loss, and the image authenticity detection model is trained using the target loss function until the training stops when the stopping condition is met, thus obtaining the trained image authenticity detection model.

12. The method according to claim 11, characterized in that, The method further includes: When the sample image is a real image, a first reconstruction loss is constructed based on the difference between the sample image and the corresponding frequency domain predicted reconstruction image, and a second reconstruction loss is constructed based on the difference between the sample image and the corresponding color predicted reconstruction image. Determining the target loss function based on the classification loss includes: The target loss function is determined by using the first reconstruction loss, the second reconstruction loss, and the classification loss.

13. A method for training an image authenticity detection model, characterized in that, The method includes: Obtain the sample image set and the sample label corresponding to each sample image in the sample image set; A frequency domain prediction missing map is obtained based on partial frequency domain information in the sample image, and a color prediction missing map is obtained based on partial color information in the sample image; A frequency domain prediction reconstruction map is generated based on the first prediction feature in the frequency domain prediction missing map, and a color prediction reconstruction map is generated based on the second prediction feature in the color prediction missing map. Based on the difference between the frequency domain prediction reconstruction map and the sample image, a frequency domain prediction mask is determined, and the first prediction feature is subjected to self-attention processing through the frequency domain prediction mask to obtain a frequency domain prediction forgery feature. Based on the difference between the color prediction reconstruction map and the sample image, a color prediction mask is determined, and the second prediction feature is subjected to self-attention processing through the color prediction mask to obtain the color prediction fake feature; By combining the frequency domain prediction forgery features and the color prediction forgery features, the authenticity prediction result of the sample image is obtained, and the classification loss is determined based on the authenticity prediction result and the corresponding sample label. The target loss function is determined based on the classification loss, and the image authenticity detection model is trained using the target loss function until the training stops when the stopping condition is met, thus obtaining the trained image authenticity detection model; wherein, the trained image authenticity detection model is used to perform image authenticity detection on the image to be detected, and obtain the corresponding authenticity detection results.

14. The method according to claim 13, characterized in that, The method further includes: When the sample image is a real image, a first reconstruction loss is constructed based on the difference between the sample image and the corresponding frequency domain predicted reconstruction image, and a second reconstruction loss is constructed based on the difference between the sample image and the corresponding color predicted reconstruction image. Determining the target loss function based on the classification loss includes: The target loss function is determined by using the first reconstruction loss, the second reconstruction loss, and the classification loss.

15. The method according to claim 14, characterized in that, The method further includes: Determine a first sample image pair and a second sample image pair in the sample image set; the first sample image pair includes two sample images with the same true / false category; the second sample image pair includes two sample images with different true / false categories; Determine the first image distance between the first predicted features corresponding to the two sample images in the first sample image pair, and the second image distance between the second predicted features corresponding to the two sample images; Determine the third image distance between the first predicted features corresponding to each of the two sample images in the second sample image pair, and the fourth image distance between the second predicted features corresponding to each of the two sample images; The first constraint loss is determined based on the first image distance and the third image distance; The second constraint loss is determined based on the second image distance and the fourth image distance; The step of determining the target loss function using the first reconstruction loss, the second reconstruction loss, and the classification loss includes: The target loss function is determined by using the first constraint loss, the second constraint loss, the first reconstruction loss, the second reconstruction loss, and the classification loss.

16. An image authenticity detection device, characterized in that, The device includes: The image reconstruction generation module is used to acquire an image to be detected, and to obtain a frequency domain missing map based on some frequency domain information in the image to be detected, and to obtain a color missing map based on some color information in the image to be detected; to generate a frequency domain reconstruction map based on a first image feature in the frequency domain missing map, and to generate a color reconstruction map based on a second image feature in the color missing map. The suspected feature determination module is used to determine a frequency domain difference mask based on the difference between the frequency domain reconstructed image and the image to be detected, and to perform self-attention processing on the first image features using the frequency domain difference mask to obtain frequency domain suspected forgery features; and to determine a color difference mask based on the difference between the color reconstructed image and the image to be detected, and to perform self-attention processing on the second image features using the color difference mask to obtain color suspected forgery features. The result output module is used to combine the frequency domain suspected forgery features and the color suspected forgery features to determine the authenticity detection result of the image to be detected.

17. A training device for an image authenticity detection model, characterized in that, The device includes: The prediction reconstruction map generation module is used to obtain a sample image set and the sample label corresponding to each sample image in the sample image set; obtain a frequency domain prediction missing map based on partial frequency domain information in the sample images, and obtain a color prediction missing map based on partial color information in the sample images; generate a frequency domain prediction reconstruction map based on the first prediction feature in the frequency domain prediction missing map, and generate a color prediction reconstruction map based on the second prediction feature in the color prediction missing map. The prediction feature determination module is used to determine a frequency domain prediction mask based on the difference between the frequency domain prediction reconstruction map and the sample image, and to perform self-attention processing on the first prediction feature through the frequency domain prediction mask to obtain a frequency domain prediction forgery feature; and to determine a color prediction mask based on the difference between the color prediction reconstruction map and the sample image, and to perform self-attention processing on the second prediction feature through the color prediction mask to obtain a color prediction forgery feature. The training module is used to integrate the frequency domain prediction forgery features and the color prediction forgery features to obtain the authenticity prediction results of the sample image, and to determine the classification loss based on the authenticity prediction results and the corresponding sample labels; to determine the target loss function based on the classification loss, and to train the image authenticity detection model through the target loss function until the training stops when the stopping condition is met, thus obtaining the trained image authenticity detection model; wherein, the trained image authenticity detection model is used to perform image authenticity detection on the image to be detected, and obtain the corresponding authenticity detection results.

18. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 15.

19. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 15.

20. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 15.

Images