A method and apparatus for image tampering detection

Abstract

The application discloses a kind of image tampering detection method and device, it is related to image detection technical field.The specific embodiment of the method for image tampering detection includes: in response to receiving target image, the target image is carried out multiple convolution operation, and preprocessed image is obtained;The preprocessed image is input into the encoder set in advance, and the encoding image is obtained;The encoding image is input into the decoder set in advance, and the corresponding inference result is obtained, and the inference result is generated The image tampering detection result corresponding to the target image of the target image is generated.The embodiment can extract the multiple image features included in target image, improve the accuracy of image tampering detection.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to a method and apparatus for detecting image tampering. Background Technology

[0002] Images are information carriers. When verifying user information, users are often required to upload images containing information, and the system then verifies the user's information based on the uploaded image. For example, in qualification certification, users need to upload images containing information such as business licenses and asset certificates. However, images are easily tampered with, which means that the information contained in the image may not be reliable. Therefore, it is necessary to detect whether images have been tampered with. Currently, commonly used image tampering detection methods mainly focus on detecting tampering in images containing natural scenes. There are fewer methods for detecting tampering in images containing text information. Image tampering detection typically includes steps such as feature extraction, contour recognition, and color comparison.

[0003] In the process of realizing this invention, the inventors discovered that the prior art has at least the following problems:

[0004] Typically, images containing text information have insufficiently obvious features and involve relatively small areas of modification, making it impossible for existing technologies to accurately determine whether an image has been tampered with. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and apparatus for image tampering detection, which can extract multiple image features included in the target image and improve the accuracy of image tampering detection.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a method for detecting image tampering is provided, comprising:

[0007] In response to receiving a target image, multiple convolution operations are performed on the target image to obtain a preprocessed image;

[0008] The preprocessed image is input into a pre-set encoder to obtain an encoded image;

[0009] The encoded image is input into a pre-set decoder to obtain the corresponding inference result, and an image tampering detection result corresponding to the target image is generated based on the inference result.

[0010] Optionally, multiple convolution operations are performed on the target image to obtain a preprocessed image, including:

[0011] The target image is convolved according to a pre-set first convolution kernel to obtain a first convolution image;

[0012] The target image is convolved according to a pre-set second convolution kernel to obtain a second convolution image;

[0013] The second convolutional image is truncated according to a pre-set truncation function to obtain the third convolutional image;

[0014] The second convolutional image is standardized to obtain the fourth convolutional image;

[0015] The fourth convolutional image is truncated according to the truncation function to obtain the fifth convolutional image;

[0016] The first convolutional image, the third convolutional image, and the fifth convolutional image are processed to obtain a preprocessed image.

[0017] Optionally, the encoder includes a four-layer network; after inputting the preprocessed image into a pre-set encoder, the method further includes:

[0018] Obtain the encoding result of the third layer network of the encoder;

[0019] The encoded result is input into a pre-set auxiliary network to obtain an auxiliary image, and the combination of the encoder, the decoder and the auxiliary network is used as the target neural network.

[0020] The first loss result corresponding to the auxiliary image and the second loss result corresponding to the image tampering detection result are determined according to a pre-set set of loss functions.

[0021] The target neural network is optimized based on the first loss result and the second loss result.

[0022] Optionally, the method further includes:

[0023] The preprocessed image is classified using hard sample mining techniques to obtain multiple types of classified images, and the target neural network is iteratively trained based on these multiple types of classified images.

[0024] Optionally, iteratively training the target neural network based on the various types of classification images includes:

[0025] The target neural network is trained multiple times based on the various types of classification images. After each training, the classification image is enlarged, and the parameter weights of the target neural network are updated.

[0026] Optionally, after determining the first loss result corresponding to the auxiliary image and the second loss result corresponding to the image tampering detection result based on a pre-set set of loss functions, the method further includes:

[0027] The loss functions included in the set of loss functions are optimized using a pre-configured optimizer.

[0028] Optionally, generating an image tampering detection result corresponding to the target image based on the inference result includes:

[0029] The reasoning result is determined according to a pre-set determination pattern to obtain a determination result;

[0030] The target image is marked according to the determination result, and an image tampering detection result corresponding to the target image is generated.

[0031] According to a second aspect of the present invention, an apparatus for detecting image tampering is provided, comprising:

[0032] The convolution module is used to perform multiple convolution operations on the target image in response to receiving the target image to obtain a preprocessed image;

[0033] The encoding module is used to input the preprocessed image into a pre-set encoder to obtain an encoded image;

[0034] The decoding module is used to input the encoded image into a pre-set decoder to obtain the corresponding inference result, and generate the image tampering detection result corresponding to the target image based on the inference result.

[0035] Optionally, multiple convolution operations are performed on the target image to obtain a preprocessed image, including:

[0036] The target image is convolved according to a pre-set first convolution kernel to obtain a first convolution image;

[0037] The target image is convolved according to a pre-set second convolution kernel to obtain a second convolution image;

[0038] The second convolutional image is truncated according to a pre-set truncation function to obtain the third convolutional image;

[0039] The second convolutional image is standardized to obtain the fourth convolutional image;

[0040] The fourth convolutional image is truncated according to the truncation function to obtain the fifth convolutional image;

[0041] The first convolutional image, the third convolutional image, and the fifth convolutional image are processed to obtain a preprocessed image.

[0042] Optionally, the encoder includes a four-layer network; the device further includes:

[0043] The acquisition module is used to acquire the encoding result of the third layer network of the encoder;

[0044] An auxiliary module is used to input the encoding result into a pre-set auxiliary network to obtain an auxiliary image, and to use the combination of the encoder, the decoder and the auxiliary network as the target neural network.

[0045] The determination module is used to determine the first loss result corresponding to the auxiliary image and the second loss result corresponding to the image tampering detection result according to a pre-set set of loss functions;

[0046] The first optimization module optimizes the target neural network based on the first loss result and the second loss result.

[0047] Optionally, the device further includes:

[0048] The classification module is used to classify the preprocessed image using hard sample mining techniques to obtain multiple types of classified images, and to iteratively train the target neural network based on the multiple types of classified images.

[0049] Optionally, iteratively training the target neural network based on the various types of classification images includes:

[0050] The target neural network is trained multiple times based on the various types of classification images. After each training, the classification image is enlarged, and the parameter weights of the target neural network are updated.

[0051] Optionally, the device further includes:

[0052] The second optimization module is used to optimize the loss functions included in the loss function set using a pre-set optimizer.

[0053] Optionally, generating an image tampering detection result corresponding to the target image based on the inference result includes:

[0054] The reasoning result is determined according to a pre-set determination pattern to obtain a determination result;

[0055] The target image is marked according to the determination result, and an image tampering detection result corresponding to the target image is generated.

[0056] According to a third aspect of the present invention, an electronic device is provided, comprising:

[0057] One or more processors;

[0058] Storage device for storing one or more programs.

[0059] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above embodiments.

[0060] According to a fourth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0061] One embodiment of the above invention has the following advantages or beneficial effects: Performing multiple convolution operations on the target image to obtain a preprocessed image can effectively extract various feature information included in the target image, improving the accuracy and efficiency of image tampering detection; using an encoder to encode the target image to obtain a well-represented encoded image, and a decoder to capture multiple features of the encoded image, achieving good decoding results; calculating the loss result between the auxiliary image and the image tampering detection result, and optimizing the target neural network based on the loss result, can improve the efficiency and generalization ability of image tampering detection, making the image tampering detection method applicable to various scenarios; classifying the preprocessed image based on the loss result to obtain various types of classified images, and iterative training based on the classified images can optimize the image tampering detection process and improve the accuracy of image tampering detection; using a pre-set optimizer to optimize the loss function set including the loss function can effectively reduce the gap between the loss result and the true result, improving the accuracy of image tampering detection; using a progressive training method when training the target neural network allows the target neural network to gradually learn the feature information of the target image, improving the detection effect of image tampering detection.

[0062] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0063] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0064] Figure 1 This is a schematic diagram of the main flow of the image tampering detection method according to an embodiment of the present invention;

[0065] Figure 2 This is a schematic diagram of the overall flow of an image tampering detection method according to a possible embodiment of the present invention;

[0066] Figure 3 This is a schematic diagram of the convolution kernel for an image tampering detection method according to a reference embodiment of the present invention;

[0067] Figure 4 This is a schematic diagram of the main flow of an image tampering detection method according to a possible embodiment of the present invention;

[0068] Figure 5 This is a schematic diagram of the main flow of an image tampering detection method according to another applicable embodiment of the present invention;

[0069] Figure 6 This is a schematic diagram of the main flow of an image tampering detection method according to another possible embodiment of the present invention;

[0070] Figure 7 This is a schematic diagram of the main modules of an image tampering detection device according to an embodiment of the present invention;

[0071] Figure 8 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0072] Figure 9 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0073] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0074] It should be noted that the collection, use, storage, sharing and transfer of user personal information involved in the technical solution of the present invention all comply with the provisions of relevant laws and regulations, and require notification to users and obtaining their consent or authorization. When applicable, user personal information is subjected to de-identification and / or anonymization and / or encryption technical processing.

[0075] Images are information carriers. When verifying user information, users are often required to upload images containing information, and the system then verifies the user's information based on the uploaded image. For example, during qualification certification, users need to upload images containing information such as business licenses and asset certificates. However, with the development of technology, image tampering has become increasingly easy, making it difficult for humans to distinguish whether an image has been tampered with, and thus difficult to determine whether the information contained in the image is accurate. Therefore, image tampering detection is necessary. Currently, commonly used image tampering detection methods mainly focus on detecting images containing natural scenes. There are fewer methods for detecting tampering in images containing text information or with complex backgrounds. Image tampering detection typically includes steps such as feature extraction, contour recognition, and color comparison.

[0076] Typically, images containing text information have less obvious features and involve smaller areas of modification. Existing technologies cannot accurately determine whether an image has been tampered with. Furthermore, format conversion is required before recognizing various types of images, leading to the loss of image information and making it impossible to accurately determine whether an image has been tampered with.

[0077] In view of this, according to a first aspect of the present invention, a method for detecting image tampering is provided.

[0078] Figure 1 This is a schematic diagram illustrating the main flow of an image tampering detection method according to an embodiment of the present invention. Figure 1 As shown, the image tampering detection method according to an embodiment of the present invention mainly includes the following steps S101 to S103.

[0079] Step S101: In response to receiving the target image, perform multiple convolution operations on the target image to obtain a preprocessed image.

[0080] The target image is the image that needs to be tampered with. For example, after the execution subject of this embodiment receives an image uploaded by the user containing information such as business license, asset certificate, contract, and agreement, the received image is used as the target image for image tampering detection. Multiple convolution operations are performed on the target image. These operations include: multiple convolution operations based on different convolution kernels, truncation using a pre-set truncation function, and addition / subtraction operations between multiple convolutional images. For example: the original image is continuously convolved with multiple pre-set convolution kernels to obtain a continuous convolutional image, which is then truncated using a pre-set truncation function to obtain a preprocessed image; or the original image is convolved with convolution kernels of different sizes and weights to obtain multiple convolutional images, which are then added or subtracted to obtain a preprocessed image; or the original image is continuously convolved with multiple pre-set convolution kernels, and a target convolutional image is selected from the multiple convolutional images obtained after multiple convolution operations, truncated using a truncation function, and then added or subtracted to the truncated target convolutional images to obtain a preprocessed image.

[0081] To enhance the robustness of the execution entity in this embodiment of the invention, data augmentation is performed on the target image before performing multiple convolution operations. For example, the target image may be flipped horizontally or vertically, randomly rotated (e.g., rotated clockwise by 90, 180, or 270 degrees), cropped according to a pre-set image size, or segmented into multiple sub-images of equal size (i.e., a portion of the target image) after cropping. Data augmentation strengthens the image features of the target image. Standardizing and normalizing the target image, performing multiple convolutions on the data-augmented target image to obtain a pre-processed image, and training a neural network based on the pre-processed image all improve the robustness and stability of the execution entity in this embodiment of the invention.

[0082] Figure 2 This is a schematic diagram illustrating the overall flow of an image tampering detection method according to a reference embodiment of the present invention. For example, as shown... Figure 2 As shown, after acquiring the target image, multiple convolution operations are performed on the target image to obtain a preprocessed image, and the obtained preprocessed image is input into the encoder (i.e., the ConvNeXt network model) for image encoding.

[0083] For another example, the target image is img1. A pre-set convolution kernel A1 is used to perform a convolution operation on the target image to obtain a convolutional image img2. A pre-set convolution kernel A2 is used to perform a convolution operation on the target image to obtain a convolutional image img3. A pre-set truncation function is used to truncate the convolutional image img2 to obtain a truncated image img4. The same truncation function is used to truncate the convolutional image img3 to obtain a truncated image img5. The truncated image img5 is subtracted from the truncated image img4 to obtain the preprocessed image (img5-img4).

[0084] Performing multiple convolution operations on the target image to obtain a preprocessed image can extract various image features from the target image, which facilitates subsequent detection of whether the preprocessed image has been tampered with, thereby improving the accuracy of image tampering detection. Before performing multiple convolution operations, data augmentation on the target image can improve the robustness and stability of the execution subject of this invention embodiment.

[0085] According to a possible embodiment of the present invention, performing multiple convolution operations on the target image to obtain a preprocessed image includes: performing a convolution operation on the target image according to a pre-set first convolution kernel to obtain a first convolution image; performing a convolution operation on the target image according to a pre-set second convolution kernel to obtain a second convolution image; truncating the second convolution image according to a pre-set truncation function to obtain a third convolution image; standardizing the data of the second convolution image to obtain a fourth convolution image; truncating the fourth convolution image according to the truncation function to obtain a fifth convolution image; and performing operations on the first convolution image, the third convolution image, and the fifth convolution image to obtain a preprocessed image.

[0086] Two convolutional kernels are pre-set: a first convolutional kernel and a second convolutional kernel. A non-zero weight value is set at the target position of the first convolutional kernel, and the weight values ​​at the remaining positions of the first convolutional kernel are set to zero. The target position of the first convolutional kernel (i.e., the position where the non-zero weight value is located) is symmetrical about the diagonal of the first convolutional kernel. Preferably, all non-zero weight values ​​set at the target position of the first convolutional kernel are equal, for example, all non-zero weight values ​​set at the target position of the first convolutional kernel are 1. A non-zero weight value is set at the target position of the second convolutional kernel. The non-zero weight values ​​of the second convolutional kernel are symmetrical left and right. The non-zero weight values ​​set at the target position of the second convolutional kernel include positive and negative numbers, decimals and integers. Preferably, the positive and negative numbers are arranged alternately, that is, when the non-zero weight value is positive, the non-zero weight values ​​at the four positions above, below, left, and right of the non-zero weight value are all negative.

[0087] Specifically, the first and second convolutional kernels are each composed of multiple sub-convolutional kernels. To ensure the first convolutional kernel is symmetrical about its diagonal, the sub-convolutional kernels located on the diagonal of the first convolutional kernel are symmetrical about their own diagonals, and the sub-convolutional kernels on either side of the diagonal of the first convolutional kernel are mutually symmetrical about the diagonal of the first convolutional kernel. To ensure the second convolutional kernel is symmetrical about its left and right sides, the sub-convolutional kernels located on the center line of the second convolutional kernel are symmetrical about its left and right sides, and the sub-convolutional kernels on either side of the center line of the second convolutional kernel are mutually symmetrical about the center line of the second convolutional kernel. The number of sub-convolutional kernels in both the first and second convolutional kernels should be square integers, for example, the first convolutional kernel consists of 4 sub-convolutional kernels and the second convolutional kernel consists of 9 sub-convolutional kernels. The sub-convolutional kernels have the same size, for example, 9*9, 16*16, or 25*25, etc. Symmetrical sub-convolutional kernels have the same weight distribution, while asymmetric sub-convolutional kernels can have the same weight distribution (e.g., the first convolutional kernel has multiple sub-convolutional kernels with all zero weights), or they can have different weight distributions. Specifically, the weight distribution of each sub-convolutional kernel included in the first convolutional kernel is symmetrical about its own diagonal, while the weight distribution of each sub-convolutional kernel included in the second convolutional kernel is either horizontally symmetrical or vertically symmetrical.

[0088] The first convolutional image is obtained by performing a convolution operation on the target image using the first convolutional kernel, and the second convolutional image is obtained by performing a convolution operation on the target image using the second convolutional kernel.

[0089] The second convolutional image is standardized to obtain the fourth convolutional image. For example, the values ​​included in the second convolutional image are rounded according to a preset number of bits to retain, or the single-precision floating-point numbers included in the second convolutional image are converted to double-precision floating-point numbers, or the floating-point numbers included in the second convolutional image are converted to integers, and so on.

[0090] A truncation function is pre-defined, symmetrical about the coordinate axes, preferably symmetrical about the origin of the coordinate axes. The truncation function is divided into three parts by two fixed points: the first part extends from negative infinity on the X-axis to the first fixed point on the X-axis; the second part extends from the first fixed point on the X-axis to the second fixed point on the X-axis; and the third part extends from the second fixed point on the X-axis to positive infinity on the X-axis. The values ​​of the truncation function in the first and third parts are constants, while the value in the second part varies with the independent variable. Preferably, the values ​​in the first and third parts are opposites of each other. The truncation function is used to truncate the second convolutional image to obtain the third convolutional image; the truncation function is also used to truncate the fourth convolutional image to obtain the fifth convolutional image. If the data included in the convolutional image is less than the value indicated by the first fixed point, the data included in the convolutional image is converted into the fixed value corresponding to the first fixed point. If the data included in the convolutional image is greater than the value indicated by the second fixed point, the data included in the convolutional image is converted into the fixed value corresponding to the second fixed point. If the data included in the convolutional image is greater than or equal to the value indicated by the first fixed point and less than or equal to the value indicated by the second fixed point, the conversion result is calculated based on the data included in the convolutional image.

[0091] Operations are performed on the first convolutional image, the third convolutional image, and the fifth convolutional image. For example, the first convolutional image is subtracted from the third convolutional image, and then the fifth convolutional image is subtracted; or the third convolutional image is added to the fifth convolutional image, and then the first convolutional image is subtracted, etc. The result of the operation is used as the preprocessed image.

[0092] Figure 3 This is a schematic diagram of a convolutional kernel for an image tampering detection method according to a reference embodiment of the present invention. Exemplarily, as shown... Figure 3 As shown, the first convolution kernel is filter1, which includes convolution kernel filter2 and convolution kernel filter3. The weight values ​​of most regions of the first convolution kernel filter1 are zero. The first convolution kernel filter1 is symmetrical about the diagonal, and all non-zero weight values ​​set in the first convolution kernel are equal (i.e., all are 1). Specifically, the first convolution kernel filter1 is obtained by arranging sub-convolution kernels filter2 and filter3 in a certain order. Sub-convolution kernel filter2 is located on the diagonal of the first convolution kernel filter1, and the remaining positions are sub-convolution kernels filter3. Sub-convolution kernels filter2 and filter3 have the same size but different weights. Sub-convolution kernel filter2 is a 7*7 non-all-zero matrix, and sub-convolution kernel filter3 is a 7*7 all-zero matrix.

[0093] like Figure 3As shown, the second convolutional kernel is filter4, which includes convolutional kernels filter5, filter6, and filter7. The non-zero weights of the second convolutional kernel filter4 are symmetrical. These non-zero weights include positive and negative numbers, decimals and integers, arranged alternately. Specifically, when a non-zero weight is positive, the non-zero weights at the top, bottom, left, and right positions are all negative. The second convolutional kernel filter4 is obtained by arranging sub-convolutional kernels filter5, filter6, and filter7 in a specific order. The second convolutional kernel filter4 is then processed by the sub-convolutional kernels filter5 from top to bottom. The sub-kernels r5, filter6, and filter7 are divided into three parts. The first part consists of three horizontally arranged sub-kernels filter5, the second part consists of three horizontally arranged sub-kernels filter6, and the third part consists of three horizontally arranged sub-kernels filter7. Sub-kernels filter5, filter6, and filter7 have the same size but different weights. The non-zero weight values ​​of sub-kernels filter5, filter6, and filter7 include positive and negative numbers, as well as decimals and integers. The positive and negative numbers are arranged alternately. That is, when a non-zero weight value is positive, the non-zero weight values ​​at the four positions above, below, left, and right of that non-zero weight value are all negative.

[0094] like Figure 3 As shown, the truncation function f(x) is a function of x. Specifically, x is the data value included in the convolution image. The truncation function f(x) consists of three parts: when x is less than -2, the truncation function f(x) equals -2; when x is greater than or equal to -2 and less than or equal to 2, the truncation function f(x) equals x; and when x is greater than 2, the truncation function f(x) equals 2.

[0095] Using multiple convolution kernels to perform multiple convolution operations on the target image to obtain a preprocessed image can improve the flexibility of the convolution operation process and extract multiple image features from the target image. Setting a truncation function can further improve the flexibility of the convolution operation process and enable the preprocessed image to include multiple image features.

[0096] By arranging and combining multiple sub-convolutional kernels of the same size but different weight distributions to generate the first and second convolutional kernels, the flexibility of the first and second convolutional kernels can be improved, and the convolution effect and efficiency of multiple convolution operations can be improved.

[0097] Step S102: Input the preprocessed image into a pre-set encoder to obtain an encoded image.

[0098] After obtaining the preprocessed image, it is input into a pre-set encoder. The encoder comprises four layers, each with different sizes and weights in its training parameters. These four layers sequentially encode the preprocessed image. The preprocessed image is input into the first layer of the encoder, and the output of each layer becomes the input of the next layer. The output of the fourth layer is used as the encoded image. Specifically, a convolutional neural network model is used as the encoder in this embodiment of the invention; preferably, a ConvNeXt network model is used.

[0099] For example, such as Figure 2 As shown, the ConvNeXt network model is used as the encoder. The ConvNeXt network model consists of four layers. The preprocessed image is input from the first layer of the network for image encoding. The output of the first layer is used as the input of the second layer, and so on. The output of the third layer of the ConvNeXt network model is the input of the auxiliary network. The encoded image is the output of the fourth layer of the ConvNeXt network model, which is also the input of the decoder.

[0100] Encoding the preprocessed image using a pre-set encoder can yield obvious image features corresponding to the target image, thereby improving the efficiency of image feature extraction.

[0101] Step S103: Input the encoded image into a pre-set decoder to obtain the corresponding inference result, and generate the image tampering detection result corresponding to the target image based on the inference result.

[0102] After obtaining the encoded image, a decoder is used to decode it. Specifically, local and global feature capture is performed on the encoded image, and the decoded features are fused with the preprocessed image to obtain the inference result. A dual attention network model is used as the network corresponding to the decoder; preferably, a DANet (Dual Attention Network) model is used. For example, the encoded image generated by the fourth layer of the encoder is input into the dual attention network model, which includes a positional attention module and a channel attention module. In the channel attention module, the encoded image is pooled using various pooling algorithms (e.g., average pooling, max pooling, etc.), and the result is transformed through a multilayer perceptron network. An activation function is then applied to the transformed result to obtain the encoded image in the channel attention module. In the positional attention module, the encoded image is pooled using various pooling algorithms, and the results are concatenated to obtain a stitched image. This stitched image is then convolved through a convolutional layer to obtain the encoded image in the positional attention module. The results from the positional attention module and the channel attention module are fused to obtain the inference result.

[0103] The inference result is a two-channel matrix, meaning each pixel in the target image corresponds to two channels. For example, the first channel of a pixel represents the probability that the pixel has been tampered with, and the second channel represents the probability that the pixel has not been tampered with. The size of the inference result is the same as that of the target image. The regions corresponding to the pixels identified as tampered with are marked on the target image to obtain the image tampering detection result for the target image.

[0104] For example, such as Figure 2 As shown, the DANet network model is used as the decoder, and the output of the fourth layer of the encoder (i.e., the encoded image) is used as the input of the decoder. The DANet network model is used to capture local and global features of the encoded image to obtain the inference result. If the probability that a pixel has been tampered with is greater than or equal to the probability that the pixel has not been tampered with, the pixel is determined to have been tampered with. The pixels determined to have been tampered with are marked on the target image, and thus the target image with the marked tampered area is obtained, which is the image tampering detection result corresponding to the target image.

[0105] Using a convolutional neural network as an encoder and a dual attention network as a decoder can improve the efficiency and accuracy of image tampering detection.

[0106] According to a possible embodiment of the present invention, the encoder includes a four-layer network; after inputting the preprocessed image into a pre-set encoder, the method further includes: obtaining the encoding result of the third layer network of the encoder; and inputting the encoding result into a pre-set auxiliary network to obtain an auxiliary image. The auxiliary network is used to assist in training the encoding result of the third layer network of the encoder. For example, when the auxiliary network is a convolutional neural network, convolution operations are performed on the encoding result of the third layer network of the encoder. Specifically, a fully convolutional network is used as the auxiliary network, preferably an FCN (Full Convolutional Network) network.

[0107] The combination of encoder, decoder and auxiliary network is used as the target neural network; the first loss result corresponding to the auxiliary image and the second loss result corresponding to the image tampering detection result are determined according to the pre-set loss function set; the loss function set includes a variety of loss functions, and multiple loss functions are set in advance within the loss function set. The loss difference corresponding to the auxiliary image and the loss difference corresponding to the image tampering detection result are calculated using multiple loss functions respectively, resulting in multiple loss differences corresponding to the auxiliary image and multiple loss differences corresponding to the image tampering detection result.

[0108] Multiple loss differences are input into a pre-set loss result calculation formula to obtain a first loss result corresponding to the auxiliary image and a second loss result corresponding to the image tampering detection result. Specifically, the loss result calculation formula is a combination of linear functions, where each term is the product of the loss difference obtained from a loss function and the loss weight corresponding to that loss function; each loss function corresponds to a loss weight, which is pre-configured or obtained through multi-task learning; for example, multiple loss weights are set to values ​​of the same order of magnitude. After obtaining the first loss result corresponding to the auxiliary image, backpropagation is performed on the encoder and auxiliary network of the target neural network based on the first loss result, adjusting the parameter weights of the encoder and auxiliary network; after obtaining the second loss result corresponding to the image tampering detection result, backpropagation is performed on the decoder of the target neural network based on the second loss result, adjusting the parameter weights of the decoder; thus, the target neural network is optimized based on the first and second loss results.

[0109] For example, such as Figure 2As shown, after obtaining the image tampering detection result, multiple pre-set loss functions are used to calculate the loss difference of the image tampering detection result. These loss functions include: cross-entropy loss function, LovaszLoss loss function, etc. The second loss result corresponding to the image tampering detection result is generated based on the loss difference calculated by the multiple loss functions. The parameter weights of the ConvNeXt network model are optimized based on the second loss result.

[0110] For another example, the pre-set loss functions include: cross-entropy loss function and Lovasz Loss loss function. The pre-set loss result calculation formula is Loss = w1*L1 + w2*L2, where Loss is the loss result of multiple losses, L1 is the loss difference calculated according to the cross-entropy loss function, w1 is the loss weight corresponding to the cross-entropy loss function, L2 is the loss difference calculated according to the Lovasz Loss loss function, and w2 is the loss weight corresponding to the Lovasz Loss loss function. The first loss result corresponding to the auxiliary image is calculated according to the loss result calculation formula.

[0111] By setting multiple loss functions and calculating multiple losses, the accuracy and efficiency of image tampering detection can be improved when the tampered area of ​​the target image is small. This avoids excessive differences between multiple loss values ​​and improves the optimization efficiency of the target neural network.

[0112] According to another possible embodiment of the present invention, the method further includes: classifying the preprocessed image using a hard sample mining technique to obtain multiple types of classified images, and iteratively training the target neural network based on the multiple types of classified images.

[0113] The image tampering detection result obtained from the preprocessed image indicates an inference region, representing an area in the preprocessed image that is likely to have been tampered with. The true result indicates a true region, representing an area in the preprocessed image that has indeed been tampered with. In a reference hard sample mining technique, a pre-set Cross-Union Ratio (CIRR) threshold is used. The CIRR represents the area of ​​overlap between the inference region and the true region divided by the total area of ​​the inference result and the true region. The CIRR is between 0 and 1; the closer it is to 1, the larger the overlap area between the inference region and the true region, and the more accurate the inference result. When the CIRR is greater than or equal to the CIRR threshold, the corresponding preprocessed image is used as a positive classification image; when the CIRR is less than the CIRR threshold, the corresponding preprocessed image is used as a negative classification image. The identified negative classification images are then input into the target neural network for iterative training. When there are a large number of negative classification images, to improve the iterative training effect of the target neural network, positive classification images are also input into the target neural network and participate in iterative training.

[0114] In another possible hard sample mining technique, multiple regions of interest (ROIs) are pre-divided on the preprocessed image. The loss difference between these ROIs is calculated, and the ROIs are sorted in descending order of loss difference to obtain a negative classification image set. Then, the ratio of the number of ROIs in each preprocessed image to the number of preprocessed images used for image tampering detection is calculated to determine the number of negative classification images required. Based on this number, the negative classification images with the largest loss differences are selected from the negative classification image set. These negative classification images are then fed into the target neural network for iterative training. Hard sample mining techniques can include OHEM (Online Hard Example Mining).

[0115] For example, the pre-set crossover ratio (CUP) threshold is 0.7, the area of ​​the inference result is 4, the area of ​​the real region is 2, the overlapping area between the inference region and the real region is 2, and the total area of ​​the inference result and the real region is 4. Dividing the overlapping area by the total area gives a CUP of 0.5. Since the CUP is less than the CUP threshold, the corresponding preprocessed image is used as the negative classification image and is continued to be input into the target neural network for iterative training.

[0116] By using hard sample mining techniques to classify preprocessed images, multiple types of classified images are obtained. Iterative training based on these multiple classified images can optimize the image tampering detection process and improve the accuracy of image tampering detection.

[0117] According to another possible embodiment of the present invention, iterative training of the target neural network based on the multiple types of classification images includes: training the target neural network multiple times based on the multiple types of classification images, wherein, after each training is completed, the size of the classification image is enlarged, and the parameter weights included in the target neural network are updated.

[0118] The target neural network is progressively trained through multiple iterations using various types of classification images. Specifically, after training the target neural network with different types of classification images, new parameter weights are obtained. Before starting the next training iteration, the parameter weights of the target neural network are modified to the new parameter weights obtained in the previous training, i.e., training continues based on the parameter weights obtained in the previous training. Furthermore, before starting the next training iteration, the image size of the classification image used in the previous training iteration is adjusted. Specifically, the image size of the classification image is increased according to a pre-set magnification ratio, so that during iterative training, the parameter weights used in the current training iteration are the parameter weights obtained in the previous training iteration, and the classification image used in the current training iteration is obtained by magnifying the classification image used in the previous training iteration.

[0119] For example, during the first training, the parameter weights used are W1, and the size of the negative classification image is "640*640". After the first training, a new parameter weight W2 is obtained. Before the second training, the size of the negative classification image is enlarged to "1024*1024", and the parameter weights of the target neural network are updated to W2. The second training is performed based on the enlarged negative classification image and the updated parameter weights. The preparation work for subsequent iterations of training is carried out in the same way.

[0120] When training the target neural network, a progressive training method is used, which allows the target neural network to gradually learn the feature information of negative classification images, thereby improving the detection effect of image tampering detection.

[0121] According to another possible embodiment of the present invention, after determining the first loss result corresponding to the auxiliary image and the second loss result corresponding to the image tampering detection result based on a pre-set loss function set, the method further includes: optimizing the loss functions included in the loss function set using a pre-set optimizer.

[0122] The gradient of the parameters included in each loss function within the set of loss functions is calculated using a pre-configured optimizer. For example, the gradient can be calculated using stochastic gradient descent. Based on the historical gradients of the parameters, the corresponding first-order momentum and second-order momentum are calculated. The formula for calculating the first-order momentum is, for example: m1t = v1 * m1t-1 + (1 - v1) * gt, where v1 represents the parameter, m1t-1 represents the first-order momentum calculated for the parameter in the previous iteration, and gt represents the gradient of the parameter. The formula for calculating the second-order momentum is, for example: m2t = v2 * m2t-1 + (1 - v2) * g. t2, where v2 represents the parameter, m2t-1 represents the second-order momentum calculated for the parameter in the previous iteration, and gt represents the gradient of the parameter; then, the descent gradient of the parameter is calculated, for example, by multiplying the corresponding gradient using a pre-set learning rate to obtain the descent gradient of the parameter; the parameters included in the loss function are optimized based on the descent gradient, with an optimization formula for example: pt+1=pt-q*m1t / m2t*1 / 2, where pt represents the value of the parameter before optimization, q represents the learning rate, and m1t and m2t represent the first-order momentum and second-order momentum, respectively. Preferably, the pre-set optimizer can be: Adam optimizer, AdamW optimizer, etc.

[0123] Using a pre-configured optimizer to optimize the parameters of each loss function within the loss function set can effectively reduce the loss result of the loss function calculation and improve the accuracy of image tampering detection.

[0124] According to a possible embodiment of the present invention, generating an image tampering detection result corresponding to the target image based on the inference result includes: judging the inference result according to a pre-set judgment mode to obtain a judgment result; and marking the target image according to the judgment result to generate an image tampering detection result corresponding to the target image.

[0125] After obtaining the inference result, a pre-set judgment mode is acquired. The judgment mode indicates how to determine whether each pixel in the inference result has been tampered with. For example, the judgment mode compares the probability values ​​corresponding to the first and second channels of the same pixel, and takes the inference result corresponding to the channel with the larger probability value as the inference result of that pixel. For example, if the probability value of the first channel (not tampered with) of the target pixel is 0.4 and the probability value of the second channel (tampered with) of the target pixel is 0.8, then the target pixel is judged to have been tampered with. Or, the judgment mode considers only the probability value of a single channel. According to a pre-set probability threshold, if the probability value of the channel is greater than or equal to the probability threshold, the inference result corresponding to that channel is taken as the inference result of that pixel. If the probability value of the channel is less than the probability threshold, the inference result corresponding to the other channel is taken as the inference result of that pixel. For example, only the probability value of the first channel (not tampered with) is judged. If the probability value of the first channel of the target pixel is 0.9 and the pre-set probability threshold is 0.7, then the target pixel is judged to have not been tampered with.

[0126] The judgment result corresponding to the inference result is obtained based on the judgment mode. The judgment result includes the tampering detection result of each pixel, that is, each pixel is judged to have been tampered with or not. The pixels judged to have been tampered with are marked on the target image, and the marked area is taken as the tampered area of ​​the target image, thereby obtaining the image tampering detection result corresponding to the target image.

[0127] By determining whether a pixel has been tampered with according to a pre-set judgment mode and marking the pixels that have been tampered with, the efficiency, flexibility and accuracy of image tampering detection can be improved.

[0128] Figure 4 This is a schematic diagram of the main flow of an image tampering detection method according to a possible embodiment of the present invention. Figure 4 As shown, the image tampering detection method may include:

[0129] Step S401: In response to receiving the target image, perform a convolution operation on the target image according to a pre-set first convolution kernel to obtain a first convolution image;

[0130] Step S402: Perform convolution operation on the target image according to the preset second convolution kernel to obtain the second convolution image;

[0131] Step S403: The second convolutional image is truncated according to a pre-set truncation function to obtain the third convolutional image;

[0132] Step S404: Standardize the data of the second convolutional image to obtain the fourth convolutional image;

[0133] Step S405: Perform the truncation function on the fourth convolutional image to obtain the fifth convolutional image;

[0134] Step S406: Perform operations on the first convolutional image, the third convolutional image, and the fifth convolutional image to obtain a preprocessed image;

[0135] Step S407: Input the preprocessed image into a pre-set encoder to obtain an encoded image;

[0136] Step S408: Input the encoded image into a pre-set decoder to obtain the corresponding inference result;

[0137] Step S409: Generate the image tampering detection result corresponding to the target image based on the reasoning result;

[0138] Step S410: Obtain the encoding result of the third layer network of the encoder, input the encoding result into the pre-set auxiliary network, and obtain the auxiliary image;

[0139] Step S411: Determine the first loss result corresponding to the auxiliary image and the second loss result corresponding to the image tampering detection result according to the pre-set loss function set;

[0140] Step S412: Optimize the encoder and auxiliary network based on the first loss result;

[0141] Step S413: Optimize the encoder and decoder based on the second loss result.

[0142] The specific implementation details of the image tampering detection method of the present invention described above have been explained in detail in the above-described image tampering detection method, so the details will not be repeated here.

[0143] Figure 5 This is a schematic diagram of the main flow of an image tampering detection method according to another possible embodiment of the present invention. Figure 5 As shown, the image tampering detection method may include:

[0144] Step S501: In response to receiving the target image, perform multiple convolution operations on the target image to obtain a preprocessed image;

[0145] Step S502: Input the preprocessed image into the ConvNeXt network model to obtain the encoded image;

[0146] Step S503: Input the encoded image into the DANet network model to obtain the inference result;

[0147] Step S504: Determine the reasoning result according to the preset determination mode to obtain the determination result;

[0148] Step S505: Mark the target image according to the judgment result and generate the image tampering detection result corresponding to the target image;

[0149] Step S506: Obtain the encoding result of the third layer of the ConvNeXt network model, input the encoding result into the pre-set FCN network model, and obtain the auxiliary image;

[0150] Step S507: Determine the first loss result corresponding to the auxiliary image and the second loss result corresponding to the image tampering detection result according to the pre-set loss function set;

[0151] Step S508: Optimize the ConvNeXt network model, DANet network model, and FCN network model based on the first loss result and the second loss result;

[0152] Step S509: Use hard sample mining technology to classify the preprocessed image to obtain various types of classified images;

[0153] Step S510: Iteratively train the target neural network based on various types of classification images. After each training, enlarge the size of the classification image and update the parameter weights of the target neural network.

[0154] Step S511: Optimize the loss functions included in the loss function set using a pre-set optimizer.

[0155] The specific implementation details of the image tampering detection method of another applicable embodiment of the present invention have been described in detail in the above-described image tampering detection method, so the details will not be repeated here.

[0156] Figure 6 This is a schematic diagram of the main flow of an image tampering detection method according to another possible embodiment of the present invention. Figure 6 As shown, the image tampering detection method may include:

[0157] Step S601: In response to receiving the target image, perform multiple convolution operations on the target image to obtain a preprocessed image;

[0158] Step S602: Input the preprocessed image into a pre-set encoder to obtain an encoded image;

[0159] Step S603: Input the encoded image into a pre-set decoder to obtain the corresponding inference result;

[0160] Step S604: Determine whether the pixels included in the reasoning result have been tampered with according to the preset determination mode, and obtain the determination result;

[0161] Step S605: The pixels determined to have been tampered with are taken as target pixels, and the target pixels are marked on the target image to generate the image tampering detection result corresponding to the target image.

[0162] Step S606: Obtain the encoding result of the third layer network of the encoder, input the encoding result into the pre-set auxiliary network, and obtain the auxiliary image;

[0163] Step S607: Determine the first loss result corresponding to the auxiliary image and the second loss result corresponding to the image tampering detection result according to the pre-set loss function set;

[0164] Step S608: Optimize the encoder and auxiliary network based on the first loss result;

[0165] Step S609: Optimize the encoder and decoder based on the second loss result;

[0166] Step S610: Optimize the cross-entropy loss function and LovaszLoss loss function included in the loss function set using a pre-set optimizer;

[0167] Step S611: Use OHEM technology to classify the preprocessed image to obtain multiple types of classified images, and iteratively train the target neural network based on the multiple types of classified images.

[0168] The specific implementation details of the image tampering detection method of another possible embodiment of the present invention have been described in detail in the above-described image tampering detection method, so the content will not be repeated here.

[0169] According to a second aspect of the present invention, an apparatus for detecting image tampering is provided.

[0170] Figure 7 This is a schematic diagram of the main modules of an image tampering detection device according to an embodiment of the present invention, as shown below. Figure 7 As shown, the image tampering detection device 700 mainly includes:

[0171] Convolution module 701 is used to perform multiple convolution operations on the target image in response to receiving the target image to obtain a preprocessed image;

[0172] Encoding module 702 is used to input the preprocessed image into a pre-set encoder to obtain an encoded image;

[0173] The decoding module 703 is used to input the encoded image into a pre-set decoder to obtain the corresponding inference result, and generate the image tampering detection result corresponding to the target image based on the inference result.

[0174] According to a possible embodiment of the present invention, multiple convolution operations are performed on the target image to obtain a preprocessed image, including:

[0175] The target image is convolved according to a pre-set first convolution kernel to obtain a first convolution image;

[0176] The target image is convolved according to a pre-set second convolution kernel to obtain a second convolution image;

[0177] The second convolutional image is truncated according to a pre-set truncation function to obtain the third convolutional image;

[0178] The second convolutional image is standardized to obtain the fourth convolutional image;

[0179] The fourth convolutional image is truncated according to the truncation function to obtain the fifth convolutional image;

[0180] The first convolutional image, the third convolutional image, and the fifth convolutional image are processed to obtain a preprocessed image.

[0181] According to another possible embodiment of the present invention, the encoder includes a four-layer network; the image tampering detection device 700 further includes:

[0182] The acquisition module is used to acquire the encoding result of the third layer network of the encoder;

[0183] An auxiliary module is used to input the encoding result into a pre-set auxiliary network to obtain an auxiliary image, and to use the combination of the encoder, the decoder and the auxiliary network as the target neural network.

[0184] The determination module is used to determine the first loss result corresponding to the auxiliary image and the second loss result corresponding to the image tampering detection result according to a pre-set set of loss functions;

[0185] The first optimization module optimizes the target neural network based on the first loss result and the second loss result.

[0186] According to yet another possible embodiment of the present invention, the image tampering detection apparatus 700 further includes:

[0187] The classification module is used to classify the preprocessed image using hard sample mining techniques to obtain multiple types of classified images, and to iteratively train the target neural network based on the multiple types of classified images.

[0188] According to another possible embodiment of the present invention, iterative training of the target neural network based on the various types of classification images includes:

[0189] The target neural network is trained multiple times based on the various types of classification images. After each training, the classification image is enlarged, and the parameter weights of the target neural network are updated.

[0190] According to another possible embodiment of the present invention, the image tampering detection apparatus 700 further includes:

[0191] The second optimization module is used to optimize the loss functions included in the loss function set using a pre-set optimizer.

[0192] According to a possible embodiment of the present invention, generating an image tampering detection result corresponding to the target image based on the inference result includes:

[0193] The reasoning result is determined according to a pre-set determination pattern to obtain a determination result;

[0194] The target image is marked according to the determination result, and an image tampering detection result corresponding to the target image is generated.

[0195] It should be noted that the specific implementation details of the image tampering detection device described in the embodiments of the present invention have been described in detail in the image tampering detection method described above, so the details will not be repeated here.

[0196] According to the technical solution of the present invention, a preprocessed image is obtained by performing multiple convolution operations on the target image, which can effectively extract various feature information included in the target image and improve the accuracy and efficiency of image tampering detection; an encoder is used to encode the target image to obtain an encoded image with good feature representation, and the decoder can capture multiple features of the encoded image to achieve good decoding effect; the loss result of the auxiliary image and the image tampering detection result is calculated, and the target neural network is optimized according to the loss result, which can improve the efficiency and generalization ability of image tampering detection, making the image tampering detection method applicable to various scenarios; the preprocessed image is classified according to the loss result to obtain various types of classified images, and iterative training is performed according to the classified images to optimize the image tampering detection process and improve the accuracy of image tampering detection; the loss function included in the loss function set is optimized using a pre-set optimizer, which can effectively reduce the gap between the loss result and the true result and improve the accuracy of image tampering detection; a progressive training method is used when training the target neural network, so that the target neural network gradually learns the feature information of the target image and improves the detection effect of image tampering detection.

[0197] According to a third aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the present invention.

[0198] According to a fourth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method provided in the first aspect of the present invention.

[0199] Figure 8 An exemplary system architecture 800 is shown, in which the image tampering detection method or apparatus of embodiments of the present invention can be applied.

[0200] like Figure 8 As shown, system architecture 800 may include terminal devices 801, 802, and 803, a network 804, and a server 805. Network 804 serves as the medium for providing communication links between terminal devices 801, 802, and 803 and server 805. Network 804 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0201] Users can use terminal devices 801, 802, and 803 to interact with server 805 via network 804 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 801, 802, and 803, such as model training applications, image tampering detection applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0202] Terminal devices 801, 802, and 803 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0203] Server 805 can be a server providing various services, such as a backend management server (for example only) supporting image tampering detection requests sent by upstream terminal devices 801, 802, and 803. In response to receiving a target image, the backend management server can perform multiple convolution operations on the target image to obtain a preprocessed image; input the preprocessed image into a pre-set encoder to obtain an encoded image; input the encoded image into a pre-set decoder to obtain a corresponding inference result; generate an image tampering detection result corresponding to the target image based on the inference result; and feed back the image tampering detection status (for example only) to the terminal devices.

[0204] It should be noted that the image tampering detection method provided in this embodiment of the invention is generally executed by server 805, and correspondingly, the image tampering detection device is generally installed in server 805. The image tampering detection method provided in this embodiment of the invention can also be executed by terminal devices 801, 802, and 803, and correspondingly, the image tampering detection device can be installed in terminal devices 801, 802, and 803.

[0205] It should be understood that Figure 8 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0206] The following is for reference. Figure 9 It shows a schematic diagram of the structure of a computer system 900 suitable for implementing a terminal device of the present invention. Figure 9 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0207] like Figure 9As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 902 or programs loaded from storage section 908 into random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the system 900. The CPU 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0208] The following components are connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 910 as needed so that computer programs read from it can be installed into storage section 908 as needed.

[0209] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by central processing unit (CPU) 901, it performs the functions defined above in the system of the embodiments of this invention.

[0210] It should be noted that the computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the embodiments of the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the embodiments of the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0211] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer programs according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0212] The modules described in the embodiments of the present invention can be implemented in software or in hardware. The described modules can also be located in a processor. For example, a processor can be described as including a convolution module, an encoding module, and a decoding module. The names of these modules do not necessarily limit the module itself. For example, the convolution module can also be described as "a module that performs multiple convolution operations on a target image".

[0213] In another aspect, embodiments of the present invention also provide a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the device, the device implements the following method: in response to receiving a target image, performing multiple convolution operations on the target image to obtain a preprocessed image; inputting the preprocessed image into a pre-set encoder to obtain an encoded image; inputting the encoded image into a pre-set decoder to obtain a corresponding inference result; and generating an image tampering detection result corresponding to the target image based on the inference result.

[0214] According to the technical solution of the present invention, a preprocessed image is obtained by performing multiple convolution operations on the target image, which can effectively extract various feature information included in the target image and improve the accuracy and efficiency of image tampering detection; an encoder is used to encode the target image to obtain an encoded image with good feature representation, and the decoder can capture multiple features of the encoded image to achieve good decoding effect; the loss result of the auxiliary image and the image tampering detection result is calculated, and the target neural network is optimized according to the loss result, which can improve the efficiency and generalization ability of image tampering detection, making the image tampering detection method applicable to various scenarios; the preprocessed image is classified according to the loss result to obtain various types of classified images, and iterative training is performed according to the classified images to optimize the image tampering detection process and improve the accuracy of image tampering detection; the loss function included in the loss function set is optimized using a pre-set optimizer, which can effectively reduce the gap between the loss result and the true result and improve the accuracy of image tampering detection; a progressive training method is used when training the target neural network, so that the target neural network gradually learns the feature information of the target image and improves the detection effect of image tampering detection.

[0215] The specific embodiments described above do not constitute a limitation on the scope of protection of the embodiments of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the embodiments of the present invention should be included within the scope of protection of the embodiments of the present invention.

Claims

1. A method for detecting image tampering, characterized in that, include: In response to receiving a target image, multiple convolution operations are performed on the target image to obtain a preprocessed image; The preprocessed image is input into a pre-set encoder to obtain an encoded image; The encoded image is input into a pre-set decoder to obtain the corresponding inference result, and an image tampering detection result corresponding to the target image is generated based on the inference result. Performing multiple convolution operations on the target image to obtain a preprocessed image includes: performing a convolution operation on the target image according to a pre-set first convolution kernel to obtain a first convolution image; performing a convolution operation on the target image according to a pre-set second convolution kernel to obtain a second convolution image; truncating the second convolution image according to a pre-set truncation function to obtain a third convolution image; standardizing the data of the second convolution image to obtain a fourth convolution image; and truncating the fourth convolution image according to the truncation function to obtain a fifth convolution image. The image is processed by subtracting the third convolutional image from the first convolutional image and then subtracting the fifth convolutional image, or by adding the fifth convolutional image to the third convolutional image and then subtracting the first convolutional image. The preprocessed image is obtained by: subtracting the third convolutional image from the first convolutional kernel and then subtracting the fifth convolutional image from the second convolutional kernel; where: the weight distribution of each sub-convolutional kernel in the first convolutional kernel is symmetrical about its own diagonal; the weight distribution of each sub-convolutional kernel in the second convolutional kernel is symmetrical about the left and right or about the top and bottom; the truncation function is symmetrical about the origin of the coordinate axes; the truncation function is divided into three parts by two fixed points; and the values ​​of the truncation function in the first and third parts are opposites of each other.

2. The method according to claim 1, characterized in that, The encoder comprises a four-layer network; after inputting the preprocessed image into the pre-set encoder, the method further includes: Obtain the encoding result of the third layer network of the encoder; The encoded result is input into a pre-set auxiliary network to obtain an auxiliary image, and the combination of the encoder, the decoder and the auxiliary network is used as the target neural network. The first loss result corresponding to the auxiliary image and the second loss result corresponding to the image tampering detection result are determined according to a pre-set set of loss functions. The target neural network is optimized based on the first loss result and the second loss result.

3. The method according to claim 2, characterized in that, The method further includes: The preprocessed image is classified using hard sample mining techniques to obtain multiple types of classified images, and the target neural network is iteratively trained based on these multiple types of classified images.

4. The method according to claim 3, characterized in that, Iterative training of the target neural network based on the various types of classified images includes: The target neural network is trained multiple times based on the various types of classification images. After each training, the classification image is enlarged, and the parameter weights of the target neural network are updated.

5. The method according to claim 4, characterized in that, After determining the first loss result corresponding to the auxiliary image and the second loss result corresponding to the image tampering detection result based on a pre-set set of loss functions, the method further includes: The loss functions included in the set of loss functions are optimized using a pre-configured optimizer.

6. The method according to claim 1, characterized in that, Based on the inference result, an image tampering detection result corresponding to the target image is generated, including: The reasoning result is determined according to a pre-set determination pattern to obtain a determination result; The target image is marked according to the determination result, and an image tampering detection result corresponding to the target image is generated.

7. An apparatus for detecting image tampering, characterized in that, include: The convolution module is used to perform multiple convolution operations on the target image in response to receiving the target image to obtain a preprocessed image; The encoding module is used to input the preprocessed image into a pre-set encoder to obtain an encoded image; The decoding module is used to input the encoded image into a pre-set decoder to obtain the corresponding inference result, and generate the image tampering detection result corresponding to the target image based on the inference result; Performing multiple convolution operations on the target image to obtain a preprocessed image includes: performing a convolution operation on the target image according to a pre-set first convolution kernel to obtain a first convolution image; performing a convolution operation on the target image according to a pre-set second convolution kernel to obtain a second convolution image; truncating the second convolution image according to a pre-set truncation function to obtain a third convolution image; standardizing the data of the second convolution image to obtain a fourth convolution image; and truncating the fourth convolution image according to the truncation function to obtain a fifth convolution image. The image is processed by subtracting the third convolutional image from the first convolutional image and then subtracting the fifth convolutional image, or by adding the fifth convolutional image to the third convolutional image and then subtracting the first convolutional image. The preprocessed image is obtained by: subtracting the third convolutional image from the first convolutional kernel and then subtracting the fifth convolutional image from the second convolutional kernel; where: the weight distribution of each sub-convolutional kernel in the first convolutional kernel is symmetrical about its own diagonal; the weight distribution of each sub-convolutional kernel in the second convolutional kernel is symmetrical about the left and right or about the top and bottom; the truncation function is symmetrical about the origin of the coordinate axes; the truncation function is divided into three parts by two fixed points; and the values ​​of the truncation function in the first and third parts are opposites of each other.

8. An electronic device, characterized in that, include: One or more processors; Storage device, used to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Images