Image tampering detection model establishment method and device and image tampering detection method
By constructing an image tamper detection model, combining RGB and DCT domain feature fusion, and dynamically controlling image compression quality factor, the problem of cumbersome and insufficient robustness of image tamper detection process in the prior art is solved, and high accuracy and robustness are achieved Sexual image tamper detection.
Patent Information
- Application Number
- CN202411077131.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-12
AI Technical Summary
The existing image tamper detection method is complicated and not robust enough, and it leads to global information loss when cropping pictures, lacks context information, and cannot effectively extract global features.
Build an image tamper detection model including RGB feature extraction module, artifact learning module, artifact feature extraction module, feature fusion module and segmentation module. By dynamically controlling the image compression quality factor, obtaining and constructing training samples from the sample set, and using training samples The image tamper detection model is trained, the features of the image RGB domain and the DCT domain are fused, and a heat map representing the image tampering information is generated.
It significantly improves the accuracy and robustness of image tamper detection, can effectively capture the overall structure of the image, enhances the model's robustness and cross-domain generalization ability of different images, and reduces the difficulty and time-consuming of audits.
Smart Images

Figure CN118941806A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence, and in particular to an image tampering detection method and device based on deep learning. Background Art
[0002] An existing image tampering detection method includes: enhancing each input image to obtain a group of homologous images; performing adaptive brightness contrast adjustment on each image; passing a group of adjusted homologous images through a trained machine learning model, and obtaining an output probability map for each image; in each output probability map, the value of each pixel represents the probability that the pixel in the homologous image is a tampered pixel; restoring each output probability map corresponding to each homologous image to its original form through the opposite operation when the homologous image is obtained, averaging the values of the same pixels in a group of output probability maps restored to their original form, and all pixel sets determined to be tampered pixels of the group of homologous images as a whole are used as the tampered area located for each input image before enhancement during testing. This image tampering detection and positioning method requires image enhancement during testing to obtain a group of homologous images, and also requires adaptive brightness contrast adjustment for each image. The process is cumbersome and lacks robustness, and only uses the RGB pixel information of the image, which is insufficient for the forensic technology of image tampering.
[0003] Another existing image tampering detection method includes: extracting spatial domain feature information and frequency domain feature information of the image to be detected, inputting the spatial domain feature information and the frequency domain feature information into the first image tampering detection network, and obtaining a first candidate tampering area; extracting edge feature information of the image to be detected, calculating the probability value of each pixel in the image to be detected belonging to the tampering area, and determining the area in the image to be detected whose probability value is greater than the corresponding probability threshold as the second candidate tampering area; inputting the spatial domain feature information, the frequency domain feature information and the edge feature information into the second image tampering detection network to obtain a third candidate tampering area; determining the tampering area of the image to be detected according to at least two of the first candidate tampering area, the second candidate tampering area and the third candidate tampering area. This image tampering detection method requires cropping the center of the input image to a fixed size, which will result in the loss of global information, the failure to extract global features, and the lack of context information, which is very unfavorable for the image tampering detection task. In addition, cropping the image will result in multiple reasoning and final splicing, and the process is too cumbersome. Summary of the invention
[0004] The present invention is used to solve the problems of complicated process and insufficient robustness in existing image tampering detection methods.
[0005] In order to solve the above technical problems, the first aspect of the present invention provides a method for establishing an image tampering detection model, comprising:
[0006] Construct an image tampering detection model including an RGB feature extraction module, an artifact learning module, an artifact feature extraction module, a feature fusion module and a segmentation module, wherein the RGB feature extraction module is used to extract RGB features according to an image, the artifact learning module is used to determine DCT features according to a quantization table of an image and a Y channel DCT coefficient of an image, the artifact feature extraction module is used to extract artifact features from the DCT features, the feature fusion module is used to fuse RGB features and artifact features to obtain fused features, and the segmentation module is used to generate a heat map representing image tampering information according to the fused features;
[0007] Acquire a sample set, the sample set includes multiple pairs of samples, each pair of samples includes a tampered image and a mask image representing the tampered area of the image;
[0008] The image compression quality factor is dynamically controlled to obtain and construct training samples from the sample set, and the image tampering detection model is trained using the training samples.
[0009] As a further embodiment of the present invention, dynamically controlling the image compression quality factor to obtain and construct training samples from the sample set, and using the training samples to train the image tampering detection model includes:
[0010] Determine the image compression quality factor according to the current training step number;
[0011] According to the image compression quality factor, obtaining and constructing training samples from the sample set;
[0012] Using the training samples to train the image tampering detection model, and updating the current number of training steps;
[0013] Detecting whether the image tampering detection model reaches a preset accuracy;
[0014] If not, continue to execute the steps of determining the image compression quality factor according to the current training step number and the steps thereafter;
[0015] If yes, the final image tampering detection model is stored.
[0016] As a further embodiment of the present invention, determining the image compression quality factor according to the current training step number includes:
[0017] Determining a selection range of the image compression quality factor according to the current training step number, wherein the selection range increases as the current training step number increases;
[0018] The image compression quality factor is dynamically selected from the selection range.
[0019] As a further embodiment of the present invention, determining a selection range of the image compression quality factor according to the current training step number includes:
[0020] The selection range is (max-S / T, max), where S represents the current training step number, T represents a preset constant, and max represents the maximum image compression quality factor.
[0021] As a further embodiment of the present invention, the RGB feature extraction module adopts a four-layer down-sampling high-resolution network, and the artifact feature extraction module adopts a three-layer down-sampling high-resolution network;
[0022] The feature fusion module is used to fuse the features of the same size in the RGB feature extraction module and the artifact feature extraction module, and to upsample and splice the fused features of each size to obtain the fused features.
[0023] As a further embodiment of the present invention, the artifact learning module includes: a DCT coefficient conversion unit, a convolution unit, a repetitive processing unit, a multiplication unit, a first shape adjustment and dimension exchange unit, a second shape adjustment and dimension exchange unit and a splicing unit;
[0024] The DCT coefficient conversion unit is used to convert the Y channel DCT coefficient to obtain a multi-dimensional DCT feature;
[0025] The convolution unit is used to perform convolution processing on the multi-dimensional DCT features to obtain intermediate DCT features;
[0026] The repeated processing unit is used to repeatedly process the quantization table of the image to obtain a processed quantization table;
[0027] The multiplication unit is used to perform multiplication processing on the processed quantization table and the intermediate DCT feature to obtain the intermediate dequantized DCT feature;
[0028] The first shape adjustment and dimension exchange unit is used to adjust the shape and dimension of the intermediate DCT features;
[0029] The second shape adjustment and dimension exchange unit is used to adjust the shape and dimension of the intermediate dequantized DCT features;
[0030] The splicing unit is used to splice the first shape adjustment and dimension exchange unit and the second shape adjustment and dimension exchange unit to obtain DCT features.
[0031] A second aspect of the present invention provides an image tampering identification method, wherein an image tampering detection model is established in advance according to the method described in any of the above embodiments, and the image tampering identification method comprises:
[0032] Receiving an image to be detected;
[0033] Inputting the image to be detected into the image tampering detection model to obtain a thermal map of the image to be detected;
[0034] The image tampering result is determined according to the thermal map of the image to be detected.
[0035] A third aspect of the present invention provides an image tampering detection model establishment device, comprising:
[0036] A model construction module, used to construct an image tampering detection model including an RGB feature extraction module, an artifact learning module, an artifact feature extraction module, a feature fusion module and a segmentation module, wherein the RGB feature extraction module is used to extract RGB features according to an image, the artifact learning module is used to determine DCT features according to a quantization table of an image and a Y channel DCT coefficient of an image, the artifact feature extraction module is used to extract artifact features from the DCT features, the feature fusion module is used to fuse RGB features and artifact features to obtain fused features, and the segmentation module is used to generate a heat map representing image tampering information according to the fused features;
[0037] A sample acquisition unit, used to acquire a sample set, the sample set includes a plurality of pairs of samples, each pair of samples includes a tampered image and a mask image representing a tampered area of the image;
[0038] A training unit is used to obtain and construct training samples from the sample set by dynamically controlling the image compression quality factor, and use the training samples to train the image tampering detection model.
[0039] A fourth aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any of the foregoing embodiments when executing the computer program.
[0040] A fifth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of a computer device, the method described in any of the foregoing embodiments is implemented.
[0041] A sixth aspect of the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor of a computer device, it implements the method described in any of the above embodiments.
[0042] The image tampering detection model establishment method and device provided by the present invention can fuse the features of the image RGB domain and the DCT domain by constructing an image tampering detection model including an RGB feature extraction module, an artifact learning module, an artifact feature extraction module, a feature fusion module and a segmentation module, thereby improving the accuracy of image tampering detection. By dynamically controlling the image compression quality factor to obtain and construct training samples from a sample set, and using the training samples to train the image tampering detection model, the image tampering detection model can be trained from easy to difficult, thereby significantly improving the robustness and cross-domain generalization ability of the image tampering detection model to different image compressions.
[0043] In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0045] Figure 1 A flow chart of a method for establishing an image tampering detection model according to an embodiment of the present invention is shown;
[0046] Figure 2 A schematic diagram of the structure of an image tampering detection model according to an embodiment of the present invention is shown;
[0047] Figure 3 A partial structural schematic diagram of an image tampering detection model according to an embodiment of the present invention is shown;
[0048] Figure 4 A structural diagram of an artifact learning module according to an embodiment of the present invention is shown;
[0049] Figure 5 A schematic diagram showing a process of training an image tampering detection model according to an embodiment of the present invention is shown;
[0050] Figure 6 A structural diagram of an apparatus for establishing an image tampering detection model according to an embodiment of the present invention is shown;
[0051] Figure 7 A flow chart of an image tampering identification method according to an embodiment of the present invention is shown;
[0052] Figure 8 A structural diagram of an image tampering identification device according to an embodiment of the present invention is shown;
[0053] Fig. 9A structural diagram of a computer device according to an embodiment of the present invention is shown.
[0054] Description of the accompanying symbols:
[0055] 201. RGB feature extraction module;
[0056] 202. Artifact learning module;
[0057] 203. Artifact feature extraction module;
[0058] 204. Feature fusion module;
[0059] 205, segmentation module;
[0060] 401, DCT coefficient conversion unit;
[0061] 402, convolution unit;
[0062] 403, repeated processing unit;
[0063] 404, multiplication unit;
[0064] 405. A first shape adjustment and dimension exchange unit;
[0065] 406. A second shape adjustment and dimension exchange unit;
[0066] 407, splicing unit;
[0067] 601, model building module;
[0068] 602. Sample acquisition unit;
[0069] 603, training unit;
[0070] 801, receiving unit;
[0071] 802, detection unit;
[0072] 803, identification unit;
[0073] 902. Computer equipment;
[0074] 904, processor;
[0075] 906. Memory;
[0076] 908, driving mechanism;
[0077] 910, input / output module;
[0078] 912. Input device;
[0079] 914. Output device;
[0080] 916. Presentation equipment;
[0081] 918. Graphical user interface;
[0082] 920, network interface;
[0083] 922, communication link;
[0084] 924. Communication bus. DETAILED DESCRIPTION
[0085] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0086] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.
[0087] This specification provides method operation steps as described in the embodiments or flow charts, but more or fewer operation steps may be included based on routine or non-creative work. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the system or device product is executed in practice, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel.
[0088] It should be noted that the image tampering detection model establishment and image tampering prediction method and device of the present invention can be used in the financial field, and can also be used in any field other than the financial field. The application field of the image tampering detection model establishment and image tampering prediction method and device of the present invention is not limited. Taking the financial field as an example, financial business processing has gradually become popular online, and the fraud risk caused by the falsification of materials (especially pictures) has also "risen with the tide". In on-site review, bank mobile account opening, credit card application, pre-loan and post-loan management, and commercial consumption scenarios, huge financial losses and potential risks are caused by fraud every year. It is difficult to manually identify whether the material image has been tampered with, and the material types are diverse and the number is large, resulting in a large workload, long time consumption, and easy omissions in the review. The present invention is applied to the above-mentioned scenarios to assist manual review and judgment, which can reduce the difficulty of the review, make the review based on evidence, and improve the review efficiency and ability.
[0089] It should be noted that the images involved in this application are information and data authorized by the user or fully authorized by all parties, and the acquisition, transmission, storage, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.
[0090] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.
[0091] In one embodiment of the present invention, a method for establishing an image tampering detection model is provided. Figure 1 As shown, including:
[0092] Step 101, constructing an image tampering detection model including an RGB feature extraction module, an artifact learning module, an artifact feature extraction module, a feature fusion module and a segmentation module.
[0093] like Figure 2 As shown, the input of the RGB feature extraction module 201 is an image, which is connected to the feature fusion module 204. The artifact learning module 202 is connected to the artifact feature extraction module 203, and the artifact feature extraction module 203 is connected to the feature fusion module 204. The feature fusion module 204 is connected to the segmentation module 205.
[0094] The RGB feature extraction module 201 is used to extract RGB features from an image.
[0095] The artifact learning module 202 is used to determine the DCT features according to the quantization table of the image and the Y channel DCT coefficient of the image, which can capture the tampering clues in the frequency domain, facilitate the extraction of tampering features of tiny visible tampering traces, and provide clues and basis for detecting image tampering. Among them, the quantization table can be obtained through the image header file.
[0096] The artifact feature extraction module 203 is used to extract artifact features from DCT features.
[0097] The feature fusion module 204 is used to fuse the RGB features and the artifact features to obtain fused features.
[0098] The segmentation module 205 is used to generate a heat map representing image tampering information based on the fused features.
[0099] In some implementations, the RGB feature extraction module 201 inputs an image size of 512*512 with 3 channels, the Y channel DCT coefficient is a 512*512 matrix, and the quantization table is an 8*8 matrix.
[0100] Step 102, obtaining a sample set, the sample set includes multiple pairs of samples, each pair of samples includes a tampered image and a mask image of the tampered area of the image. For example, the tampered area is represented by 1, and the non-tampered area is represented by 0. In this embodiment, the tampered image can be data collected from reality, and the mask image is obtained by other technologies or manual analysis of the tampered image.
[0101] Step 103 , dynamically controlling the image compression quality factor to obtain and construct training samples from the sample set, and using the training samples to train the image tampering detection model.
[0102] The image tampering detection model constructed in this embodiment can fuse the features of the image RGB domain and the DCT domain, can capture the overall structure of the image without losing the fine compression artifacts required for forensic investigation, and improve the accuracy of image tampering detection. By dynamically controlling the image compression quality factor to obtain training samples from the sample set, and using the training samples to train the image tampering detection model, the image tampering detection model can be trained from easy to difficult, thereby significantly improving the robustness of the image tampering detection model to different image compressions and the cross-domain generalization ability.
[0103] In one embodiment of the present invention, Figure 3As shown, the RGB feature extraction module 201 adopts a high-resolution network with four layers of down sampling, and the artifact feature extraction module 203 adopts a high-resolution network with three layers of down sampling. Each layer of the network of the RGB feature extraction module 201 and the artifact feature extraction module 203 includes multiple convolution units, and the number of convolutions in each layer can be set according to actual conditions. When down sampling, a stride convolution with a step size of 2 can be used to achieve it. This embodiment replaces the pooling layer with a convolution, which can realize the extraction of subtle signals and improve the accuracy of tampering feature extraction.
[0104] During specific implementation, the RGB feature extraction module 201 and the artifact feature extraction module 203 may use the HRNet network as the backbone network for extracting RGB features and artifact features.
[0105] The feature fusion module 204 is used to fuse the features of the same size in the RGB feature extraction module 201 and the artifact feature extraction module 203, and to upsample and splice the fused features of each size to obtain a fused feature.
[0106] In some embodiments, Figure 3 As shown, the number of channels obtained after the first three convolutions in the first layer of the RGB feature extraction module 201 are 64, 256, and 48 respectively, and the convolution is maintained at 48 channels. The number of channels obtained in the second layer of the RGB feature extraction module is 96. The number of channels obtained in the third layer of the RGB feature extraction module is 192, and the number of channels obtained in the fourth layer of the RGB feature extraction module is 384.
[0107] The number of channels obtained after the first two convolutions in the first layer of the artifact feature extraction module 203 are 512 and 96 respectively, and the subsequent convolutions maintain the number of channels at 96. The number of channels obtained in the second layer of the artifact feature extraction module 203 is 192. The number of channels obtained in the third layer of the artifact feature extraction module 203 is 384.
[0108] The feature fusion module 204 superimposes the features of the second to fourth layers of the RGB feature extraction module 201 with the features of the first to third layers of the artifact feature extraction module 203, and then performs channel adjustment through convolution. The first layer of features of the RGB feature extraction module 201 is convolved and downsampled, and the feature data of each channel is fused to obtain a feature with a channel of 360, and a 2-channel feature is obtained through convolution.
[0109] This embodiment can improve the accuracy of tampering prediction results by extracting multi-scale features.
[0110] In one embodiment of the present invention, Figure 4As shown, the artifact learning module 202 includes: a DCT coefficient conversion unit 401, a convolution unit 402, a repetition processing unit 403, a multiplication unit 404, a first shape adjustment and dimension exchange unit 405, a second shape adjustment and dimension exchange unit 406 and a splicing unit 407.
[0111] The DCT coefficient conversion unit 401 is used to convert the Y channel DCT coefficient to obtain a multi-dimensional DCT feature. The present invention can facilitate the subsequent extraction of DCT features and improve the resolution of DCT features through the setting of the DCT coefficient conversion unit 401.
[0112] The convolution unit 402 is used to perform convolution processing on the multi-dimensional DCT features to obtain intermediate DCT features. The convolution unit is used to perform feature and channel transformation on the multi-dimensional DCT features, including a 3×3 convolution unit and a 1×1 convolution unit, and each convolution unit also includes a batch normalization unit (batch normalization) and a linear rectification unit (ReLU). The intermediate DCT features can also be called a JPEG header.
[0113] The repetitive processing unit 403 is used to repeatedly process the quantization table of the image to obtain a processed quantization table.
[0114] The multiplication unit 404 is used to multiply the processed quantization table and the intermediate DCT feature to obtain the intermediate dequantized DCT feature. The JPEG header and the quantization table are multiplied by the multiplication unit 404 to obtain the frequency component, which is similar to the dequantization process of the DCT coefficient in JPEG decoding.
[0115] The first shape adjustment and dimension exchange unit 405 is used to adjust the shape and dimension of the intermediate DCT features.
[0116] The second reshape adjustment and dimension exchange unit 406 is used to adjust the shape and dimension of the intermediate dequantized DCT features.
[0117] The concatenation unit 407 is used to concatenate the first shape adjustment and dimension exchange unit 405 and the second shape adjustment and dimension exchange unit 406 to obtain DCT features.
[0118] In some specific implementations, a 21-dimensional multi-dimensional DCT feature is obtained through a DCT coefficient conversion unit 401, 64-dimensional data is obtained through a 3×3 convolution unit, and the number of channels can be reduced to 4 through a 1×1 convolution unit, that is, a 4-dimensional intermediate DCT feature is obtained. 256-dimensional data are obtained through a first shape adjustment and dimension exchange unit 405 and a second shape adjustment and dimension exchange unit 406, respectively, and 512-dimensional data is obtained through a splicing unit 407.
[0119] The DCT features obtained in this embodiment can well capture the discontinuities of the artifact grid, serving as another important clue for locating the tampered area, thereby making up for the shortcomings caused by the unclear visual features.
[0120] In one embodiment of the present invention, Figure 5 As shown, by dynamically controlling the image compression quality factor to obtain and construct training samples from a sample set, the image tampering detection model is trained using the training samples, including:
[0121] Step 501, determining an image compression quality factor according to the current training step number.
[0122] Specifically, the current training step number is the current training cycle. Specifically, determining the image compression quality factor according to the current training step number includes: determining a selection range of the image compression quality factor according to the current training step number, wherein the selection range increases as the current training step number increases; and dynamically selecting the image compression quality factor from the selection range.
[0123] The larger the current training step number, the larger the selection range, and the more possibilities of the corresponding image compression quality factor.
[0124] In some implementations of the present invention, determining a selection range of an image compression quality factor according to a current number of training steps includes:
[0125] The selection range is (max-S / T, max), where S represents the current training step number, T represents a preset constant, and max represents the maximum image compression quality factor.
[0126] Step 502: Acquire and construct training samples from a sample set according to the image compression quality factor.
[0127] When this step is implemented, the image in the training sample is compressed using the image compression quality factor. The specific implementation process can refer to the existing technology and will not be described in detail here.
[0128] Step 503: Use the training samples to train the image tampering detection model and update the current training step number.
[0129] When this step is implemented, each time the image tampering detection model is trained, the current training step number increases by 1.
[0130] Step 504, detect whether the image tampering detection model reaches a preset accuracy, if not, return to step 501 to continue execution, if yes, store the final image tampering detection model.
[0131] This embodiment can train the image tampering detection model from simple data to difficult data by setting the image compression quality factor, and can significantly improve the robustness and cross-domain generalization ability of the model to different image compressions.
[0132] In contrast, when the dynamic image compression quality factor is uniformly selected in a random range, the image compression quality factor selection is chaotic and cannot be well learned to extract effective image tampering features, which significantly reduces the performance and generalization ability of the model on the sample set. Therefore, it can be proved that the above image compression quality factor selection strategy can effectively enhance the robustness and generalization ability of the trained model.
[0133] Based on the same inventive concept, the present invention also provides an image tampering detection model establishment device, as described in the following embodiments. Since the principle of solving the problem by the image tampering detection model establishment device is similar to that of the image tampering detection model establishment method, the implementation of the image tampering detection model establishment device can refer to the image tampering detection model establishment method, and the repeated parts will not be repeated.
[0134] Specifically, Figure 6 As shown, the image tampering detection model establishment device includes:
[0135] A model building module 601 is used to build an image tampering detection model including an RGB feature extraction module, an artifact learning module, an artifact feature extraction module, a feature fusion module and a segmentation module, wherein the RGB feature extraction module is used to extract RGB features according to an image, the artifact learning module is used to determine DCT features according to a quantization table of an image and a Y channel DCT coefficient of an image, the artifact feature extraction module is used to extract artifact features from the DCT features, the feature fusion module is used to fuse RGB features and artifact features to obtain fused features, and the segmentation module is used to generate a heat map representing image tampering information according to the fused features;
[0136] A sample acquisition unit 602 is used to acquire a sample set, where the sample set includes multiple pairs of samples, each pair of samples includes a tampered image and a mask image representing the tampered area of the image;
[0137] The training unit 603 is used to obtain and construct training samples from the sample set by dynamically controlling the image compression quality factor, and use the training samples to train the image tampering detection model.
[0138] Based on the establishment of the image tampering detection model, image tampering can be performed, specifically, Figure 7 As shown, the image tampering identification method includes:
[0139] Step 701, receiving an image to be detected;
[0140] Step 702, inputting the image to be detected into the image tampering detection model to obtain a heat map of the image to be detected;
[0141] Step 703: Determine the image tampering result according to the thermal map of the image to be detected.
[0142] Correspondingly, in some embodiments of the present invention, Figure 8 As shown, an image tampering identification device is also provided, comprising:
[0143] The receiving unit 801 receives an image to be detected.
[0144] The detection unit 802 inputs the image to be detected into the image tampering detection model to obtain a heat map of the image to be detected.
[0145] The identification unit 803 determines the image tampering result according to the thermal map of the image to be detected.
[0146] The technical solution provided by the present invention can achieve the following technical effects:
[0147] (1) The present invention can support images of all sizes as input, does not require cropping, and can capture and extract global features.
[0148] (2) The present invention uses the fusion features of the image RGB and DCT domains to detect image tampering, and adopts an artifact learning module to generate an artifact feature map, which overcomes the limitations of other technical solutions. It has a stable and robust detection effect for secondary JPEG compression caused by image tampering, and its generalization is stronger than that of existing solutions.
[0149] (3) The training method of the image tampering detection model of the method of the present invention enables the model to learn image tampering features step by step from easy to difficult, thereby enhancing the robustness and generalization of the model.
[0150] In some embodiments of the present invention, a computer device is also provided, such as Fig. 9As shown, the computer device 902 may include one or more processors 904, such as one or more central processing units (CPUs), each of which may implement one or more hardware threads. The computer device 902 may also include any memory 906, which is used to store any kind of information such as code, settings, data, etc. Non-limitingly, for example, the memory 906 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory may use any technology to store information. Further, any memory may provide volatile or non-volatile retention of information. Further, any memory may represent a fixed or removable component of the computer device 902. In one case, when the processor 904 executes an associated instruction stored in any memory or a combination of memories, the computer device 902 may perform any operation of the associated instruction. The computer device 902 also includes one or more drive mechanisms 908 for interacting with any memory, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.
[0151] The computer device 902 may also include an input / output module 910 (I / O) for receiving various inputs (via input devices 912) and for providing various outputs (via output devices 914). A specific output mechanism may include a presentation device 916 and an associated graphical user interface 918 (GUI). In other embodiments, the input / output module 910 (I / O), input device 912, and output device 914 may not be included, and the computer device 902 may be used as a computer device in a network. The computer device 902 may also include one or more network interfaces 920 for exchanging data with other devices via one or more communication links 922. One or more communication buses 924 couple the components described above together.
[0152] The communication link 922 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 922 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.
[0153] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are executed.
[0154] An embodiment of the present invention further provides a computer-readable instruction, wherein when a processor executes the instruction, the program therein causes the processor to execute the method described in any of the aforementioned embodiments.
[0155] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0156] It should also be understood that in the embodiments of the present invention, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects before and after are in an "or" relationship.
[0157] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0158] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0159] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.
[0160] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.
[0161] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0162] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0163] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A method for establishing an image tampering detection model, characterized in that: include: Construct an image tampering detection model including an RGB feature extraction module, an artifact learning module, an artifact feature extraction module, a feature fusion module and a segmentation module, wherein the RGB feature extraction module is used to extract RGB features according to an image, the artifact learning module is used to determine DCT features according to a quantization table of an image and a Y channel DCT coefficient of an image, the artifact feature extraction module is used to extract artifact features from the DCT features, the feature fusion module is used to fuse RGB features and artifact features to obtain fused features, and the segmentation module is used to generate a heat map representing image tampering information according to the fused features; Acquire a sample set, the sample set includes multiple pairs of samples, each pair of samples includes a tampered image and a mask image representing the tampered area of the image; The image compression quality factor is dynamically controlled to obtain and construct training samples from the sample set, and the image tampering detection model is trained using the training samples.
2. The method according to claim 1, characterized in that The method of acquiring and constructing training samples from the sample set by dynamically controlling the image compression quality factor and using the training samples to train the image tampering detection model comprises: Determine the image compression quality factor according to the current training step number; According to the image compression quality factor, obtaining and constructing training samples from the sample set; Using the training samples to train the image tampering detection model, and updating the current number of training steps; Detecting whether the image tampering detection model reaches a preset accuracy; If not, continue to execute the steps of determining the image compression quality factor according to the current training step number and the steps thereafter; If yes, the final image tampering detection model is stored.
3. The method according to claim 2, characterized in that The determining of the image compression quality factor according to the current training step number includes: Determining a selection range of the image compression quality factor according to the current training step number, wherein the selection range increases as the current training step number increases; The image compression quality factor is dynamically selected from the selection range.
4. The method according to claim 3, characterized in that According to the current number of training steps, the selection range of the image compression quality factor is determined, including: The selection range is (max-S / T, max), where S represents the current training step number, T represents a preset constant, and max represents the maximum image compression quality factor.
5. The method according to claim 1, characterized in that The RGB feature extraction module uses a four-layer down-sampling high-resolution network, and the artifact feature extraction module uses a three-layer down-sampling high-resolution network; The feature fusion module is used to fuse the features of the same size in the RGB feature extraction module and the artifact feature extraction module, and to upsample and splice the fused features of each size to obtain the fused features.
6. The method according to claim 1, characterized in that The artifact learning module includes: a DCT coefficient conversion unit, a convolution unit, a repetition processing unit, a multiplication unit, a first shape adjustment and dimension exchange unit, a second shape adjustment and dimension exchange unit and a splicing unit; The DCT coefficient conversion unit is used to convert the Y channel DCT coefficient to obtain a multi-dimensional DCT feature; The convolution unit is used to perform convolution processing on the multi-dimensional DCT features to obtain intermediate DCT features; The repeated processing unit is used to repeatedly process the quantization table of the image to obtain a processed quantization table; The multiplication unit is used to perform multiplication processing on the processed quantization table and the intermediate DCT feature to obtain the intermediate dequantized DCT feature; The first shape adjustment and dimension exchange unit is used to adjust the shape and dimension of the intermediate DCT features; The second shape adjustment and dimension exchange unit is used to adjust the shape and dimension of the intermediate dequantized DCT features; The splicing unit is used to splice the first shape adjustment and dimension exchange unit and the second shape adjustment and dimension exchange unit to obtain DCT features.
7. A method for identifying image tampering, characterized in that: An image tampering detection model is established in advance according to the method according to any one of claims 1 to 6, and the image tampering identification method comprises: Receiving an image to be detected; Inputting the image to be detected into the image tampering detection model to obtain a thermal map of the image to be detected; The image tampering result is determined according to the thermal map of the image to be detected.
8. A device for establishing an image tampering detection model, characterized in that: include: A model construction module, used to construct an image tampering detection model including an RGB feature extraction module, an artifact learning module, an artifact feature extraction module, a feature fusion module and a segmentation module, wherein the RGB feature extraction module is used to extract RGB features according to an image, the artifact learning module is used to determine DCT features according to a quantization table of an image and a Y channel DCT coefficient of an image, the artifact feature extraction module is used to extract artifact features from the DCT features, the feature fusion module is used to fuse RGB features and artifact features to obtain fused features, and the segmentation module is used to generate a heat map representing image tampering information according to the fused features; A sample acquisition unit, used to acquire a sample set, the sample set includes a plurality of pairs of samples, each pair of samples includes a tampered image and a mask image representing a tampered area of the image; A training unit is used to obtain and construct training samples from the sample set by dynamically controlling the image compression quality factor, and use the training samples to train the image tampering detection model.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor of a computer device, the method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Image tampering recognition model training method and device and image tampering recognition method and device
CN111368342A
Tampered video detection method and system based on multi-domain block feature mark point registration
CN114693607A
Picture forgery detection method and system based on multivariate feature fusion segmentation
CN115393697A
Self-supervised document image tampering detection method and device and medium
CN116524294A
Cited By
JPEG compression artifact learning module and image tampering detection system
CN121121426A
