Image detection method and method for obtaining image detection model

Abstract

The present application discloses an image detection method, which is characterized by comprising: obtaining an image to be detected; inputting the image to be detected into an image detection model, and outputting detection results of real areas and forged areas in the image to be detected; wherein the image detection model is obtained by training with training samples added with transmission environment noise. The above method is used to solve the problem of forgery detection for forged images transmitted through a transmission environment.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular, to an image detection method, an image detection device, a method for obtaining an image detection model, an apparatus for obtaining an image detection model, two electronic devices, and two storage devices. Background Art

[0002] Currently, the increasing popularity of image editing tools (such as Photoshop and Meitu) has made it easier for people to modify images at the digital level. However, the modified images (i.e., forged images) are constantly threatening many fields, such as removing the original watermark, fabricating fake news, and serving as fictional evidence. At the same time, the booming development of online social networks on the Internet has made it the main channel for information dissemination. Naturally, the popularity of online social networks has also facilitated the spread of forged images, making it easier to report fake news or spread rumors on the Internet.

[0003] Therefore, the detection and localization of forged images have become extremely important and can be widely applied to filtering fake news, avoiding malicious insurance claims, verifying the authenticity of documents, etc. Currently, researchers have proposed many detection methods to identify tampered images to ensure information security. Some of these forensics aim to detect specific forms of tampering, such as splicing, copy-pasting, and inpainting, while others are used to identify more complex or fused multiple forgeries.

[0004] However, due to the following two reasons, few people have studied the detection of image forgery on online social networks. First, almost all online social networks process the uploaded images in a lossy manner, which may completely erase the forgery traces left by the original tampering operations, thus rendering the existing forensic algorithms ineffective. Second, the lossy operations adopted by online social networks are usually uncertain and not within the control of users, making it difficult for forensic algorithms to accurately simulate them.

[0005] Therefore, it is an urgent problem to detect forged images after being transmitted through a transmission environment. Summary of the Invention

[0006] The present application provides an image detection method and an image detection device to solve the problem of detecting forged images after being transmitted through a transmission environment.

[0007] The present application provides an image detection method, including:

[0008] Obtaining an image to be detected;

[0009] Input the image to be detected into an image detection model, and output the detection results of the real region and the forged region in the image to be detected; wherein, the image detection model is trained using training samples with added transmission environment noise.

[0010] As an implementation, it includes:

[0011] Adversarial noise is also added to the training samples of the image detection model.

[0012] This application also provides a method for obtaining an image detection model, including:

[0013] Add a forged image to the original image to obtain an initial sample;

[0014] Add transmission environment noise of a preset transmission environment to the initial sample to obtain a processed sample with added transmission environment noise;

[0015] Use the processed sample as a training sample and provide it to the image detection model to be trained, and perform training on it to identify the real region and the forged region.

[0016] As an implementation, after adding the transmission environment noise of the preset transmission environment to the initial sample, further add adversarial noise to obtain a processed sample with added transmission environment noise and adversarial noise.

[0017] As an implementation, adding the transmission environment noise of the preset transmission environment to the initial sample to obtain a processed sample with added transmission environment noise includes:

[0018] Provide the image serving as the initial sample to the preset transmission environment, and after transmission through the preset transmission environment, use the transmitted image as the processed sample with added transmission environment noise; or,

[0019] Construct a transmission environment simulation model that simulates the preset transmission environment, add simulated transmission environment noise τ to the image serving as the initial sample, and use the image after the above processing as the processed sample with added transmission environment noise.

[0020] As an implementation, the method for obtaining the transmission environment simulation model includes the following steps:

[0021] Provide the original image to the preset transmission environment to obtain the actually transmitted image after transmission through the preset transmission environment;

[0022] Use the original image provided to the preset transmission environment as a sample, provide it to the transmission environment simulation model to be trained, and generate a transmission environment prediction image according to the output of the transmission environment simulation model to be trained;

[0023] Compare the actually transmitted image with the predicted transmission environment image to generate an adjustment metric;

[0024] Adjust the transmission environment simulation model to be trained according to the adjustment metric until a preset adjustment target is reached.

[0025] As an implementation manner, the adjustment metric includes a loss function.

[0026] As an implementation manner, the transmission environment simulation model to be trained adopts a neural network model, and a differentiable JPEG compression convolutional layer is embedded in the neural network model

[0027] As an implementation manner, the adversarial noise is obtained in the following manner:

[0028] Given an input image x, obtain the prediction result of the transmission environment simulation model for this image, and process it to obtain a preliminary prediction noise τ o ;

[0029] According to the given input image x, the value y of the labeled forgery area, and the value of the forgery area initially output by the image detection model to be trained Preliminary prediction noise τ o , construct the adversarial noise ξ.

[0030] As an implementation manner, for the t-th given input image x, the direction of the adversarial noise ξ is set to the average gradient of the first t - 1 image samples.

[0031] The present application also provides an image detection device, including:

[0032] A to-be-detected image acquisition unit, configured to acquire a to-be-detected image;

[0033] An image detection unit, configured to input the to-be-detected image into an image detection model and output detection results of real regions and forgery regions in the to-be-detected image; wherein, the image detection model is trained using training samples with added transmission environment noise.

[0034] The present application also provides an electronic device, including:

[0035] A processor; and

[0036] A memory, configured to store a program of an image detection method. After the device is powered on and runs the program of the image detection method through the processor, the following steps are executed:

[0037] Acquire a to-be-detected image;

[0038] Input the image to be detected into an image detection model, and output the detection results of the real region and the forged region in the image to be detected; wherein, the image detection model is trained using training samples with added transmission environment noise.

[0039] This application also provides a storage device storing a program for an image detection method, which is run by a processor to execute the following steps:

[0040] Obtain an image to be detected;

[0041] Input the image to be detected into an image detection model, and output the detection results of the real region and the forged region in the image to be detected; wherein, the image detection model is trained using training samples with added transmission environment noise.

[0042] This application also provides an apparatus for obtaining an image detection model, including:

[0043] An initial sample obtaining unit for adding a forged image to an original image to obtain an initial sample;

[0044] A processed sample obtaining unit for adding transmission environment noise of a preset transmission environment to the initial sample to obtain a processed sample with added transmission environment noise;

[0045] An image detection model training unit for using the processed sample as a training sample and providing it to an image detection model to be trained, and training it to identify real regions and forged regions.

[0046] This application also provides an electronic device, which includes:

[0047] A processor; and

[0048] A memory for storing a program for a method of obtaining an image detection model. After the device is powered on and runs the program for the method of obtaining an image detection model through the processor, the following steps are executed:

[0049] Add a forged image to the original image to obtain an initial sample;

[0050] Add transmission environment noise of a preset transmission environment to the initial sample to obtain a processed sample with added transmission environment noise;

[0051] Use the processed sample as a training sample and provide it to an image detection model to be trained, and train it to identify real regions and forged regions.

[0052] This application also provides a storage device storing a program for a method of obtaining an image detection model, which is run by a processor to execute the following steps:

[0053] Add a forged image to the original image to obtain an initial sample;

[0054] Add transmission environment noise of a preset transmission environment to the initial sample to obtain a processed sample with the transmission environment noise added;

[0055] Use the processed sample as a training sample and provide it to an image detection model to be trained, and perform training on it to identify real regions and forged regions.

[0056] Compared with the prior art, the present application has the following advantages:

[0057] The present application provides an image detection method, including: obtaining an image to be detected; inputting the image to be detected into an image detection model, and outputting a detection result of real regions and forged regions in the image to be detected; wherein, the image detection model is trained using a training sample with transmission environment noise added. In the image detection method provided by the present application, since the transmission environment noise is added to the training sample during the training of the image detection model, the image detection model can well detect real regions and forged regions in the image to be detected after being transmitted through the transmission environment, improving the accuracy of forged image detection.

[0058] In a preferred solution, adversarial noise is further added to the training sample of the image detection model. Since adversarial noise is introduced during the training of the image detection model, the image detection model has stronger noise resistance, further improving the accuracy of forged image detection. Description of the Drawings

[0059] Figure 1A is a schematic diagram of a scenario embodiment provided by the present application.

[0060] Figure 1 is a flowchart of an image detection method provided by the first embodiment of the present application.

[0061] Figure 2 is a schematic diagram of an image detection method provided by an embodiment of the present application.

[0062] Figure 3 is a flowchart of a method for obtaining an image detection model provided by the second embodiment of the present application.

[0063] Figure 4 is a flowchart of a method for obtaining a transmission environment simulation model provided by the second embodiment of the present application. Detailed Description of the Invention

[0064] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0065] To enable those skilled in the art to better understand the solution of the present application, the specific application scenario embodiments of the present application will be described in detail first.

[0066] The image detection method provided in the first embodiment of the present application can be applied to scenarios where a client interacts with a server. For example, Figure 1A when it is necessary to identify the forged areas of a to-be-detected image after being transmitted through a transmission environment, usually the client first establishes a connection with the server. After the connection is established, the client sends the to-be-detected image to the server. After the server receives the to-be-detected image, the server inputs the to-be-detected image into an image detection model and outputs the detection results of the real areas and the forged areas in the to-be-detected image. Among them, the image detection model is trained using training samples with added transmission environment noise, and then the detection results of the real areas and the forged areas in the to-be-detected image are provided to the client, and the client receives the detection results of the real areas and the forged areas in the to-be-detected image.

[0067] The first embodiment of the present application provides an image detection method. The following will be described in conjunction with Figure 1 、 Figure 2 for illustration.

[0068] Step S101, obtain the to-be-detected image.

[0069] The to-be-detected image refers to an image with added transmission environment noise. The transmission environment may include application software of an online social network, etc. When a user uploads or downloads an image through the application software of an online social network, the application software of the online social network will introduce transmission environment noise during the transmission process of the image. The to-be-detected image may be an image that has been artificially processed before transmission. For example, an original image is edited using an image editing tool such as Photoshop to obtain an image containing forged areas, and the image containing forged areas generates the to-be-detected image after being transmitted through the application software of an online social network.

[0070] Step S102, input the to-be-detected image into an image detection model and output the detection results of the real areas and the forged areas in the to-be-detected image; among them, the image detection model is trained using training samples with added transmission environment noise.

[0071] The real area refers to the area in the to-be-detected image that has not been processed.

[0072] The forged area refers to the area in the image to be detected that has been processed. For example, if the original image is a grassland and an image of a car is inserted into the image using an image editing tool such as Photoshop, the inserted car image is the forged area.

[0073] When the image detection model is being trained, if the training samples include transmission environment noise under a specific transmission environment, the trained image detection model can only detect the image to be detected after being transmitted through the specific transmission environment. To make the image detection model have generalization ability, that is, it can detect images transmitted through other transmission environments, adversarial noise can be added to the training samples of the image detection model. As an implementation, adversarial noise is added to the training samples of the image detection model.

[0074] For example, as Figure 2 the detector in θ , which is an image detection model, is trained using training samples with added transmission environment noise. The forged input image is the image to be detected, and the forged input image is input into the detector f

[0075] In the image detection method provided in the first embodiment of this application, since the transmission environment noise is added to the training samples during the training of the image detection model, the image detection model can well detect the real area and the forged area in the image to be detected after being transmitted through the transmission environment, improving the accuracy of forged image detection; further, in the preferred solution, adversarial noise is added to the training samples of the image detection model to make the image detection model have generalization ability.

[0076] The second embodiment of this application provides a method for obtaining an image detection model, which will be described below in conjunction with Figure 3 , Figure 4.

[0077] In step S301, a forged image is added to the original image to obtain an initial sample.

[0078] In step S302, transmission environment noise of a preset transmission environment is added to the initial sample to obtain a processed sample with added transmission environment noise.

[0079] The preset transmission environment can be an application software of a certain online social network, or the transmission environment when uploading an image for printing, or other transmission environments.

[0080] Adding the transmission environment noise of the preset transmission environment to the initial sample to obtain a processed sample with added transmission environment noise includes:

[0081] Provide the image serving as the initial sample to a preset transmission environment. After transmission through the preset transmission environment, use the transmitted image as the processed sample with transmission environment noise added; or,

[0082] Construct a transmission environment simulation model that simulates the preset transmission environment, add simulated transmission environment noise τ to the image serving as the initial sample, and use the image after the above processing as the processed sample with transmission environment noise added.

[0083] When adding the transmission environment noise of the preset transmission environment to the initial sample to obtain the processed sample with transmission environment noise added, the image serving as the initial sample can be provided to the preset transmission environment. After transmission through the transmission environment, a processed sample with transmission environment noise added is generated. However, when the number of samples is very large, this method has relatively low efficiency. To improve the efficiency of generating the processed sample, in the second embodiment of this application, the transmission environment noise can also be simulated through a transmission environment simulation model that simulates the preset transmission environment, and the simulated transmission environment noise is added to the image of the initial sample to generate the processed sample with transmission environment noise added.

[0084] In step S303, use the processed sample as a training sample and provide it to the image detection model to be trained for training on identifying real regions and forged regions.

[0085] As an implementation, after adding the transmission environment noise of the preset transmission environment to the initial sample, adversarial noise can be further added to obtain a processed sample with transmission environment noise and adversarial noise added.

[0086] When using a transmission environment simulation model that simulates the preset transmission environment to generate a processed sample with transmission environment noise added, the second embodiment of this application further includes the following steps: Obtain the transmission environment simulation model.

[0087] Specifically, the method for obtaining the transmission environment simulation model can include the following steps:

[0088] Provide the original image to the preset transmission environment to obtain the actually transmitted image after transmission through the preset transmission environment;

[0089] Use the image transmitted through the transmission environment as a sample and provide it to the transmission environment simulation model to be trained, and generate a transmission environment prediction image according to the output of the transmission environment simulation model to be trained;

[0090] Compare the actually transmitted image with the transmission environment prediction image to generate an adjustment measurement index;

[0091] Adjust the to-be-trained transmission environment simulation network according to the adjusted measurement index until a preset adjustment target is reached.

[0092] Please refer to Figure 4 , which is a flowchart of obtaining a transmission environment simulation model provided by the second embodiment of the present application, specifically including steps S401 to S404.

[0093] Step S401: Provide the original image to a preset transmission environment to obtain an actually transmitted image after transmission through the preset transmission environment.

[0094] As Figure 2 In 2-1 of, it is a schematic diagram of obtaining a transmission environment simulation model. The original image x is obtained from the database D1, and the original image x is provided to a preset transmission environment to obtain an actually transmitted image after transmission through the preset transmission environment, which is transmitted x.

[0095] Step S402: Use the original image provided to the preset transmission environment as a sample, provide it to the to-be-trained transmission environment simulation model, and generate a transmission environment prediction image according to the output of the to-be-trained transmission environment simulation model.

[0096] The to-be-trained transmission environment simulation model uses a neural network model.

[0097] The output of the transmission environment simulation model can be the simulated transmission environment noise itself.

[0098] As Figure 2 In 2-1 of, the predicted x is the generated transmission environment prediction image.

[0099] Step S403: Compare the actually transmitted image after transmission with the transmission environment prediction image to generate an adjustment measurement index.

[0100] The adjustment measurement index is used to measure the distance between the simulated transmission environment noise and the real transmission noise. The adjustment measurement index includes: a loss function.

[0101] Step S404: Adjust the to-be-trained transmission environment simulation model according to the adjusted measurement index until a preset adjustment target is reached.

[0102] The preset adjustment target includes: an adjustment target measured by an objective function, an adjustment target measured by a regression loss function, etc.

[0103] The adversarial noise can be obtained in the following manner:

[0104] Given an input image x, obtain the prediction result of the transmission environment simulation model for this image, and process it to obtain a preliminary predicted noise τo ;

[0105] According to the given input image x, the value y of the marked forgery area, and the value of the forgery area initially output by the image detection model to be trained and the preliminary predicted noise τ o , construct the adversarial noise ξ.

[0106] The process of obtaining the preliminary predicted noise τ is introduced below o .

[0107] Let the transmission environment simulation model be g φ , where φ is the cluster of trainable parameters in the transmission environment simulation model, and the general objective function can be expressed as:

[0108]

[0109] Here, a regression loss function based on the L2 norm can be selected:

[0110]

[0111] Due to the invisibility and smallness of the transmission environment noise, in order to ensure that the transmission environment simulation model can accurately perform regression prediction on it, this application can adopt residual learning technology to convert the predicted transmitted image into a predicted residual, enabling the transmission environment simulation model to better depict the small fluctuations, that is:

[0112]

[0113] Since JPEG compression is usually adopted in the process of online social networks, in order to ensure that the regression-predicted noise also has a similar compressed noise distribution, a differentiable JPEG compression convolutional layer can be embedded in the neural network model so that the neural network can not only better fit the real propagation process but also ensure that the predicted residual has the property of compressed noise. Considering that the neural network module needs to be differentiable, this application performs a differentiable approximation on the quantization operation in the JPEG compression process. That is, the quantization operation is replaced with the following differentiable formula:

[0114]

[0115] Other processes related to JPEG can be fitted through the network. Through the above processing, the simulated preliminary predicted noise τ can be finally obtained o :

[0116]

[0117] The following introduces the process of constructing the adversarial noise ξ based on the given input image x, the value y of the labeled forgery area, the value ŷ of the forgery area initially output by the image detection model to be trained, and the initially predicted noise τ. o , for constructing the adversarial noise ξ.

[0118] From the perspective of the image detection model, noise can be decomposed into two categories: those that affect the detection effect and those that do not. Since the noise that does not affect the detection effect does not degrade the performance of the image detection model, there is no need to model it; for the noise that affects the detection effect, this application proposes to model it using learnable adversarial noise. Among them, adversarial noise is a newly emerging type of adversarial sample in recent years, aiming to use tiny perturbations to affect the judgment of deep neural networks.

[0119] Specifically, for the given input image x, the value y of the labeled forgery area, the value ŷ of the forgery area initially output by the image detection model to be trained, and the initially predicted noise τ o , the adversarial noise ξ that affects the detection effect can be designed as:

[0120]

[0121] where is the sign function, represents calculating the gradient of the cross-entropy loss function , and is:

[0122]

[0123] The adversarial noise defined above starting from the image detection model itself can greatly improve the robustness and generalization of the model.

[0124] However, the currently defined adversarial noise depends on specific input samples, thus lacking the ability to generalize to unknown samples. To comprehensively improve the generalization ability of the model, this application proposes to adjust the direction of the adversarial noise to the global gradient direction and adopt the idea of random fitting to randomly generate a noise subset from known samples. Specifically, for the t-th given input image x, the adversarial noise ξ is set as the average gradient of the previous t - 1 image samples:

[0125]

[0126] where the adversarial noise ξ is initialized to 0.

[0127] Although the above formula can be used to describe the average gradient of the input sample, it can only reflect the distribution of known samples (i.e., training data), but it is difficult to reflect the distribution of unknown samples. Therefore, this application proposes to use a parameter-containing model to further model it. In order to adopt a suitable model, 1000 random noises are first visualized by T-SNE (t-distributed stochastic neighbor embedding, a machine learning algorithm for dimensionality reduction). The visualization results show that the sampling noise points focus on a certain center point and gradually spread outward. Therefore, this application can use a Gaussian model to model adversarial noise:

[0128]

[0129] Where u is the mean of the Gaussian model defined as follows:

[0130]

[0131] σ is the experimentally determined Gaussian model variance, and ∈ is a hyperparameter used to adjust the noise intensity. So far, in actual training, the above-mentioned parameter-containing model can be used in combination with the Monte Carlo sampling method to efficiently and conveniently sample the noise ξ.

[0132] After modeling the preliminary prediction noise and anti-noise, the optimization goal of this application can be obtained:

[0133]

[0134] The output is the final image detection model, which can be used for forgery detection in real situations. The specific algorithm flow is as follows:

[0135]

[0136] The algorithm requires the following inputs: training data sets D1 and D2; training iterations N1 and N2; learning rate l during training φ and l θ .

[0137] Final output of the algorithm: trained detector f θ .

[0138] So far, the introduction of the second embodiment of this application has been completed. The second embodiment of this application proposes to use a combination of a deep neural network, residual learning, a differentiable JPEG module, etc. to simulate and characterize the transmission environment noise introduced by it. Introducing such simulated noise into the training framework can make the image detection model have excellent robustness. At the same time, based on adversarial noise, stochastic gradient method, Gaussian model, etc., this application innovatively proposes a modeling method for robust noise, which can greatly improve the performance of the image detection model. To sum up, this application is based on the preliminary prediction noise τ o and reasonable modeling of adversarial noise ξ, achieving satisfactory robustness against online social network transmission and accurate forgery detection effect.

[0139] To understand this application more clearly, the following will be combined with Figure 2 introduce an overall framework designed according to the first and second embodiments of this application.

[0140] This application proposes a forgery detection framework with high robustness. As Figure 2 shown, it mainly includes four stages: 1) training of the OSN simulation network g φ , corresponding to 2-1; 2) modeling of the transmission environment noise τ, corresponding to 2-2; 3) modeling of the unknown noise ξ, corresponding to 2-3; and 4) training of the detector f θ , corresponding to 2-4. After the above four stages, the trained detector f θ can be obtained for forgery detection in practice, corresponding to 2-5.

[0141] Corresponding to the image detection method provided in the first embodiment of this application, the third embodiment of this application provides an image detection device.

[0142] The image detection device includes:

[0143] An image to be detected obtaining unit, configured to obtain an image to be detected;

[0144] An image detection unit, configured to input the image to be detected into an image detection model and output the detection results of the real area and the forged area in the image to be detected; wherein, the image detection model is trained using training samples with added transmission environment noise.

[0145] As an implementation manner, it includes: adversarial noise is also added to the training samples of the image detection model.

[0146] It should be noted that for a detailed description of the image detection device provided in the third embodiment of this application, reference can be made to the relevant description of the first embodiment of this application, which will not be elaborated here.

[0147] Corresponding to the image detection method provided in the first embodiment of the present application, the fourth embodiment of the present application provides an electronic device, including:

[0148] A processor; and

[0149] A memory for storing a program of the image detection method. After the device is powered on and runs the program of the image detection method through the processor, the following steps are executed:

[0150] Obtain an image to be detected;

[0151] Input the image to be detected into the image detection model, and output the detection results of the real region and the forged region in the image to be detected; wherein, the image detection model is obtained by training with training samples added with transmission environment noise.

[0152] It should be noted that for the detailed description of the electronic device provided in the fourth embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, which will not be elaborated here.

[0153] Corresponding to the method for obtaining an image detection model provided in the second embodiment of the present application, the sixth embodiment of the present application provides an apparatus for obtaining an image detection model.

[0154] Obtain an image to be detected;

[0155] Input the image to be detected into the image detection model, and output the detection results of the real region and the forged region in the image to be detected; wherein, the image detection model is obtained by training with training samples added with transmission environment noise.

[0156] It should be noted that for the detailed description of the storage device provided in the fifth embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, which will not be elaborated here.

[0157] Corresponding to the method for obtaining an image detection model provided in the second embodiment of the present application, the sixth embodiment of the present application provides an apparatus for obtaining an image detection model.

[0158] The apparatus for obtaining the image detection model includes:

[0159] An initial sample obtaining unit for adding a forged image to the original image to obtain an initial sample;

[0160] A processed sample obtaining unit for adding transmission environment noise of a preset transmission environment to the initial sample to obtain a processed sample added with transmission environment noise;

[0161] An image detection model training unit, which is used to use the processed sample as a training sample and provide it to the image detection model to be trained for training to identify real regions and forged regions.

[0162] As an implementation manner, the apparatus for obtaining the image detection model further includes: an adversarial noise adding unit, which is used to add transmission environment noise of a preset transmission environment to the initial sample and then further add adversarial noise to obtain a processed sample with the transmission environment noise and adversarial noise added.

[0163] As an implementation manner, the processed sample obtaining unit is specifically used for:

[0164] Providing the image serving as the initial sample to a preset transmission environment, and after transmission through the preset transmission environment, using the transmitted image as the processed sample with the transmission environment noise added; or,

[0165] Constructing a transmission environment simulation model that simulates the preset transmission environment, adding simulated transmission environment noise τ to the image serving as the initial sample, and using the image after the above processing as the processed sample with the transmission environment noise added.

[0166] As an implementation manner, the apparatus for obtaining the image detection model further includes: a transmission environment simulation model obtaining unit, which is used for:

[0167] Providing the original image to a preset transmission environment to obtain the actually transmitted image after transmission through the preset transmission environment;

[0168] Using the image after transmission through the transmission environment as a sample, providing it to the transmission environment simulation model to be trained, and generating a transmission environment prediction image according to the output of the transmission environment simulation model to be trained;

[0169] Comparing the actually transmitted image with the transmission environment prediction image to generate an adjustment measurement index;

[0170] Adjusting the transmission environment simulation model to be trained according to the adjustment measurement index until a preset adjustment target is reached.

[0171] As an implementation manner, the adjustment measurement index includes a loss function.

[0172] As an implementation manner, the transmission environment simulation model to be trained adopts a neural network model, and a differentiable JPEG compression convolutional layer is embedded in the neural network model

[0173] As an implementation manner, the apparatus for obtaining the image detection model further includes: an adversarial noise obtaining unit, which is used for:

[0174] Given the input image \(x\), obtain the prediction result of the transmission environment simulation model for this image, and process it to obtain the preliminary prediction noise \(\tau\). o ;

[0175] According to the given input image \(x\), the labeled forged region value \(y\), and the preliminary output forged region value of the image detection model to be trained the preliminary prediction noise \(\tau\) o , construct the adversarial noise \(\xi\).

[0176] As an implementation manner, the adversarial noise obtaining unit is specifically configured to: for the \(t\)-th given input image \(x\), the adversarial noise \(\xi\) is set as the average gradient of the first \(t - 1\) image samples.

[0177] It should be noted that for the detailed description of the device provided in the sixth embodiment of the present application, reference can be made to the relevant description of the second embodiment of the present application, which will not be elaborated here.

[0178] Corresponding to the method for obtaining an image detection model provided in the second embodiment of the present application, the seventh embodiment of the present application provides an electronic device.

[0179] The electronic device includes:

[0180] a processor; and

[0181] a memory for storing a program of the method for obtaining an image detection model. After the device is powered on and runs the program of the method for obtaining an image detection model through the processor, the following steps are executed:

[0182] Add a forged image to the original image to obtain an initial sample;

[0183] Add transmission environment noise of a preset transmission environment to the initial sample to obtain a processed sample with added transmission environment noise;

[0184] Use the processed sample as a training sample and provide it to the image detection model to be trained for training to identify real regions and forged regions.

[0185] As an implementation manner, after adding transmission environment noise of a preset transmission environment to the initial sample, further add adversarial noise to obtain a processed sample with added transmission environment noise and adversarial noise.

[0186] As an implementation manner, adding transmission environment noise of a preset transmission environment to the initial sample to obtain a processed sample with added transmission environment noise includes:

[0187] Provide the image serving as the initial sample to a preset transmission environment. After transmission through the preset transmission environment, use the transmitted image as the processed sample with transmission environment noise added thereto; or,

[0188] Construct a transmission environment simulation model simulating the preset transmission environment, add simulated transmission environment noise τ to the image serving as the initial sample, and use the image after the above processing as the processed sample with transmission environment noise added thereto.

[0189] As an implementation manner, the method for obtaining the transmission environment simulation model includes the following steps:

[0190] Provide the original image to the preset transmission environment to obtain the actually transmitted image after transmission through the preset transmission environment;

[0191] Use the image transmitted through the transmission environment as a sample, provide it to the transmission environment simulation model to be trained, and generate a transmission environment prediction image according to the output of the transmission environment simulation model to be trained;

[0192] Compare the actually transmitted image with the transmission environment prediction image to generate an adjustment measurement index;

[0193] Adjust the transmission environment simulation model to be trained according to the adjustment measurement index until a preset adjustment target is reached.

[0194] As an implementation manner, the adjustment measurement index includes a loss function.

[0195] As an implementation manner, the transmission environment simulation model to be trained adopts a neural network model, and a differentiable JPEG compression convolutional layer is embedded in the neural network model

[0196] As an implementation manner, the adversarial noise is obtained in the following manner:

[0197] Given an input image x, obtain the prediction result of the transmission environment simulation model for this image, and process to obtain preliminary prediction noise τ o ;

[0198] According to the given input image x, the value y of the marked forgery area, and the value of the forgery area initially output by the image detection model to be trained preliminary prediction noise τ o , construct the adversarial noise ξ.

[0199] As an implementation manner, for the t-th given input image x, the adversarial noise ξ is set to the average gradient of the previous t - 1 image samples.

[0200] It should be noted that for the detailed description of the electronic device provided in the seventh embodiment of the present application, reference can be made to the relevant description of the second embodiment of the present application, which will not be elaborated here.

[0201] Corresponding to the method for obtaining an image detection model provided in the second embodiment of the present application, the eighth embodiment of the present application provides a storage device storing a program for the method for obtaining an image detection model. When the program is run by a processor, the following steps are executed:

[0202] Add a forged image to the original image to obtain an initial sample;

[0203] Add transmission environment noise of a preset transmission environment to the initial sample to obtain a processed sample with added transmission environment noise;

[0204] Use the processed sample as a training sample and provide it to the image detection model to be trained for training to identify real regions and forged regions.

[0205] It should be noted that for the detailed description of the storage device provided in the eighth embodiment of the present application, reference can be made to the relevant description of the second embodiment of the present application, which will not be elaborated here.

[0206] Although the present application is disclosed above with preferred embodiments, it is not intended to limit the present application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application should be defined by the scope of the claims of the present application.

[0207] In a typical configuration, a computing device includes one or more processors (CPUs), a memory-mapped input / output interface, a network interface, and a memory.

[0208] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM), and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0209] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information accessible by a computing device. As defined herein, a computer-readable medium does not include transitory media such as modulated data signals and carrier waves.

[0210] Those skilled in the art will appreciate that the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

Claims

1. An image detection method, characterized in that: include: Obtaining an image to be detected; Inputting the image to be detected into an image detection model, and outputting detection results of real areas and forged areas in the image to be detected; The method for obtaining the image detection model is as follows: adding a forged image to the original image to obtain an initial sample; The transmission environment noise of the preset transmission environment is added to the initial sample to obtain a processed sample with the transmission environment noise added. After the transmission environment noise of the preset transmission environment is added to the initial sample, adversarial noise is further added to obtain a processed sample with the transmission environment noise and adversarial noise added. The adversarial noise is obtained in the following manner: given an input image x, a prediction result of the transmission environment simulation model for the image is obtained, and a preliminary prediction noise is obtained by processing. ; Based on the given input image x, the annotated forged area value y, and the forged area value initially output by the image detection model to be trained , preliminary prediction of noise , construct the anti-noise ; For the t-th given input image x, the adversarial noise The direction of is set to the average direction of the gradient vector of the cross entropy loss function of the first t-1 training samples; The processed samples are used as training samples and provided to the image detection model to be trained to train the model to identify the real area and the forged area.

2. A method for obtaining an image detection model, characterized in that: include: Add a forged image to the original image to obtain an initial sample; The transmission environment noise of the preset transmission environment is added to the initial sample to obtain a processed sample with the transmission environment noise added. After the transmission environment noise of the preset transmission environment is added to the initial sample, adversarial noise is further added to obtain a processed sample with the transmission environment noise and adversarial noise added. The adversarial noise is obtained in the following manner: given an input image x, a prediction result of the transmission environment simulation model for the image is obtained, and a preliminary prediction noise is obtained by processing. ; Based on the given input image x, the annotated forged area value y, and the forged area value initially output by the image detection model to be trained , preliminary prediction of noise , construct the anti-noise ; For the t-th given input image x, the adversarial noise The direction of is set to the average direction of the gradient vector of the cross entropy loss function of the first t-1 training samples; The processed samples are used as training samples and provided to the image detection model to be trained to train the model to identify the real area and the forged area.

3. According to the method for obtaining the image detection model of claim 2, adding the transmission environment noise of a preset transmission environment to the initial sample to obtain a processed sample with the transmission environment noise added, comprising: Providing the image as the initial sample to a preset transmission environment, and after transmission in the preset transmission environment, using the transmitted image as the processed sample with the transmission environment noise added thereto; or A transmission environment simulation model simulating a preset transmission environment is constructed, and simulated transmission environment noise τ is added to the image as the initial sample, and the image after the above processing is used as the processed sample with the transmission environment noise added.

4. The method for obtaining the image detection model according to claim 3, characterized in that: The method for obtaining the transmission environment simulation model comprises the following steps: Providing the original image to a preset transmission environment to obtain an actual transmitted image transmitted through the preset transmission environment; Provide the original image provided to the preset transmission environment as a sample to the transmission environment simulation model to be trained, and generate a transmission environment prediction image according to the output of the transmission environment simulation model to be trained; Comparing the actual transmitted image with the transmission environment predicted image to generate an adjustment measurement index; According to the adjustment measurement index, the transmission environment simulation model to be trained is adjusted until a preset adjustment target is reached.

5. The method for obtaining an image detection model according to claim 4, characterized in that: The adjustment metric includes a loss function.

6. The method for obtaining the image detection model according to claim 5, characterized in that: The transmission environment simulation model to be trained adopts a neural network model, in which a differentiable JPEG compression convolution layer is embedded. .

7. An image detection device, characterized in that: include: An image acquisition unit to be detected, used for acquiring an image to be detected; An image detection unit, used for inputting the image to be detected into an image detection model, and outputting detection results of real areas and forged areas in the image to be detected; The method for obtaining the image detection model is as follows: adding a forged image to the original image to obtain an initial sample; The transmission environment noise of the preset transmission environment is added to the initial sample to obtain a processed sample with the transmission environment noise added. After the transmission environment noise of the preset transmission environment is added to the initial sample, adversarial noise is further added to obtain a processed sample with the transmission environment noise and adversarial noise added. The adversarial noise is obtained in the following manner: given an input image x, a prediction result of the transmission environment simulation model for the image is obtained, and a preliminary prediction noise is obtained by processing. ; Based on the given input image x, the annotated forged area value y, and the forged area value initially output by the image detection model to be trained , preliminary prediction of noise , construct the anti-noise ; For the t-th given input image x, the adversarial noise The direction of is set to the average direction of the gradient vector of the cross entropy loss function of the first t-1 training samples; The processed samples are used as training samples and provided to the image detection model to be trained to train the model to identify the real area and the forged area.

8. An electronic device, characterized in that: include: processor; as well as The memory is used to store a program of the image detection method. After the device is powered on and the program of the image detection method is run by the processor, the following steps are performed: Obtaining an image to be detected; Inputting the image to be detected into an image detection model, and outputting detection results of real areas and forged areas in the image to be detected; The method for obtaining the image detection model is as follows: adding a forged image to the original image to obtain an initial sample; The transmission environment noise of the preset transmission environment is added to the initial sample to obtain a processed sample with the transmission environment noise added. After the transmission environment noise of the preset transmission environment is added to the initial sample, adversarial noise is further added to obtain a processed sample with the transmission environment noise and adversarial noise added. The adversarial noise is obtained in the following manner: given an input image x, a prediction result of the transmission environment simulation model for the image is obtained, and a preliminary prediction noise is obtained by processing. ; Based on the given input image x, the annotated forged area value y, and the forged area value initially output by the image detection model to be trained , preliminary prediction of noise , construct the anti-noise ; For the t-th given input image x, the adversarial noise The direction of is set to the average direction of the gradient vector of the cross entropy loss function of the first t-1 training samples; The processed samples are used as training samples and provided to the image detection model to be trained to train the model to identify the real area and the forged area.

9. A storage device, characterized in that: A program for storing an image detection method is executed by a processor to perform the following steps: Obtaining an image to be detected; Inputting the image to be detected into an image detection model, and outputting detection results of real areas and forged areas in the image to be detected; The method for obtaining the image detection model is as follows: adding a forged image to the original image to obtain an initial sample; The transmission environment noise of the preset transmission environment is added to the initial sample to obtain a processed sample with the transmission environment noise added. After the transmission environment noise of the preset transmission environment is added to the initial sample, adversarial noise is further added to obtain a processed sample with the transmission environment noise and adversarial noise added. The adversarial noise is obtained in the following manner: given an input image x, a prediction result of the transmission environment simulation model for the image is obtained, and a preliminary prediction noise is obtained by processing. ; According to the given input image x, the annotated forged area value y, and the forged area value initially output by the image detection model to be trained , preliminary prediction of noise , construct the anti-noise ; For the t-th given input image x, the adversarial noise The direction of is set to the average direction of the gradient vector of the cross entropy loss function of the first t-1 training samples; The processed samples are used as training samples and provided to the image detection model to be trained to train the model to identify the real area and the forged area.

10. A device for obtaining an image detection model, characterized in that: include: An initial sample obtaining unit, used for adding a forged image into an original image to obtain an initial sample; A processed sample obtaining unit is used to add the transmission environment noise of the preset transmission environment to the initial sample to obtain the processed sample with the transmission environment noise added. After adding the transmission environment noise of the preset transmission environment to the initial sample, further add the anti-noise to obtain the processed sample with the transmission environment noise and the anti-noise added. The anti-noise is obtained in the following manner: given an input image x, the prediction result of the transmission environment simulation model for the image is obtained, and the preliminary prediction noise is obtained by processing. ; Based on the given input image x, the annotated forged area value y, and the forged area value initially output by the image detection model to be trained , preliminary prediction of noise , construct the anti-noise ; For the t-th given input image x, the adversarial noise The direction of is set to the average direction of the gradient vector of the cross entropy loss function of the first t-1 training samples; The image detection model training unit is used to provide the processed samples as training samples to the image detection model to be trained, and train it to identify real areas and forged areas.

11. An electronic device, comprising: processor; as well as The memory is used to store a program of a method for obtaining an image detection model. After the device is powered on and the program of the method for obtaining an image detection model is run by the processor, the following steps are performed: Add a forged image to the original image to obtain an initial sample; The transmission environment noise of the preset transmission environment is added to the initial sample to obtain a processed sample with the transmission environment noise added. After the transmission environment noise of the preset transmission environment is added to the initial sample, adversarial noise is further added to obtain a processed sample with the transmission environment noise and adversarial noise added. The adversarial noise is obtained in the following manner: given an input image x, a prediction result of the transmission environment simulation model for the image is obtained, and a preliminary prediction noise is obtained by processing. ; Based on the given input image x, the annotated forged area value y, and the forged area value initially output by the image detection model to be trained , preliminary prediction of noise , construct the anti-noise ; For the t-th given input image x, the adversarial noise The direction of is set to the average direction of the gradient vector of the cross entropy loss function of the first t-1 training samples; The processed samples are used as training samples and provided to the image detection model to be trained to train the model to identify the real area and the forged area.

12. A storage device, characterized in that: A program storing a method for obtaining an image detection model is executed by a processor to perform the following steps: Add a forged image to the original image to obtain an initial sample; The transmission environment noise of the preset transmission environment is added to the initial sample to obtain a processed sample with the transmission environment noise added. After the transmission environment noise of the preset transmission environment is added to the initial sample, adversarial noise is further added to obtain a processed sample with the transmission environment noise and adversarial noise added. The adversarial noise is obtained in the following manner: given an input image x, a prediction result of the transmission environment simulation model for the image is obtained, and a preliminary prediction noise is obtained by processing. ; Based on the given input image x, the annotated forged area value y, and the forged area value initially output by the image detection model to be trained , preliminary prediction of noise , construct the anti-noise ; For the t-th given input image x, the adversarial noise The direction of is set to the average direction of the gradient vector of the cross entropy loss function of the first t-1 training samples; The processed samples are used as training samples and provided to the image detection model to be trained to train the model to identify the real area and the forged area.

Images