Image tampering detection method and device

Abstract

Embodiments of the present application provide an image tampering detection method and device. The method comprises: performing noise extraction processing on a target image to obtain a noise residual graph. The noise residual graph is divided into N reference subgraphs. For any reference subgraph, the tampering score maximum subgraph is determined by a tampering score model; the tampering score maximum subgraph is one of the reference subgraph and the changed subgraph of the reference subgraph; the changed subgraph is a subgraph obtained by expanding or reducing the area of the reference subgraph in the noise residual graph. The tampering region of the target image is determined by the N tampering score maximum subgraphs corresponding to the N reference subgraphs. The area of the changed subgraph is continuously changed adaptively by the score of the tampering score model. If there is a tampering trace in the subgraphs obtained in this way, the relative proportion of small size tampering traces in the subgraph is the highest, which can effectively avoid the omission of small size tampering traces. In this way, the recognition rate of small size tampering traces is improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to an image tampering detection method, apparatus, computing device and computer-readable storage medium. Background Technology

[0002] In recent years, with the development of computer technology and image processing technology, especially with cutting-edge image processing software such as Adobe's powerful Photoshop, users can easily tamper with real images to create fake images. The widespread dissemination of these fake images online has a serious impact on our society.

[0003] Some scholars have proposed a method for detecting image tampering. The image is segmented into image patches of the same size, and then the image patches are input into LSTM (Long Short Term Memory networks) to learn the tampering correlation between different image patches. After fusing the tampering correlation between each image patch with the extracted features of the image, the result is input into a classifier to determine the tampered areas of the image.

[0004] The methods described above require identifying the tampered areas of an image through the correlation between different image blocks, making them suitable for detecting images with large tampered areas. However, for images with very small tampering marks, such as alterations to individual characters, numbers, or signatures, small tampering marks have little impact on the correlation between image blocks, making it difficult to accurately detect small tampered areas using the methods described above.

[0005] In summary, the embodiments of the present invention provide an image tampering detection method to improve the recognition rate of small-sized tampering traces. Summary of the Invention

[0006] This invention provides an image tampering detection method to improve the recognition rate of small-sized tampering traces.

[0007] In a first aspect, embodiments of the present invention provide an image tampering detection method, comprising:

[0008] The target image is subjected to noise extraction processing to obtain a noise residual map;

[0009] The noise residual map is divided into N baseline sub-maps;

[0010] For any baseline subgraph, the subgraph with the largest tampering score is determined by the tampering scoring model; the subgraph with the largest tampering score is one of the baseline subgraph and the modified subgraph of the baseline subgraph; the modified subgraph is the subgraph obtained by expanding or shrinking the region of the baseline subgraph in the noise residual map;

[0011] The tampered area of ​​the target image is determined by identifying the N sub-images with the highest tampering scores corresponding to the N reference sub-images.

[0012] Because noise fluctuations are significant at the tampering boundaries in the target image, a noise residual map is obtained by extracting noise from the target image. This residual map can then be used to derive a tampering feature score, quantifying the degree of tampering. After dividing the noise residual map into N baseline sub-images, the regions of these sub-images are adaptively changed based on the tampering scoring model. This process ultimately identifies the N sub-images with the highest tampering scores. In these sub-images, if tampering traces are present, small-sized traces will have the highest relative proportion, effectively preventing them from being missed. However, not all of the N sub-images with the highest tampering scores necessarily contain tampering traces; some may simply be normal image noise. Therefore, the tampered regions in the target image are further determined based on these N sub-images. This improves the recognition rate of small-sized tampering traces.

[0013] Optionally, the subgraph with the largest tampered score is determined by the tampering scoring model, including:

[0014] The first tampered score of the baseline subgraph is determined by the tampered scoring model, and the first tampered score is used as the tampered baseline score;

[0015] According to the set regional change rules, the changed submap of the baseline submap is determined from the noise residual map;

[0016] The second tampering score of the altered subgraph is determined using the tampering scoring model;

[0017] If the second tampering score is greater than the tampering baseline score, then the tampering baseline score is updated to the second tampering score, and the process returns to determine the changed sub-map of the baseline sub-map from the noise residual map according to the set regional change rules, until the second tampering score is no greater than the tampering baseline score;

[0018] The subgraph corresponding to the tampered benchmark score is determined as the subgraph with the largest tampered score.

[0019] By combining the tampering score of the tampering scoring model with adaptive changes to the region of the baseline sub-image, a sub-image with the highest tampering score can be determined for each baseline sub-image. In the sub-images obtained in this way, if there are tampering traces, the small-sized tampering traces will have the highest relative proportion in the sub-image, which can effectively prevent small-sized tampering traces from being missed.

[0020] Optionally, the noise residual map is divided into N reference sub-maps, including:

[0021] A sliding window is used to sample the noise residual map at a set step size to obtain N reference submaps;

[0022] According to the established regional change rules, the changed submap of the baseline submap is determined from the noise residual map, including:

[0023] According to the set regional change rules, the region of the reference sub-map is sampled in the noise residual map by expanding or shrinking the center position of the reference sub-map to obtain the changed sub-map of the reference sub-map.

[0024] By using sliding window sampling, multiple baseline sub-maps can be obtained based on the step size. A smaller step size yields more baseline sub-maps, allowing for adjustment of the accuracy of the final tampered region detection. The modified sub-map is obtained by expanding or shrinking the noise residual map, centered on the original baseline sub-map. This ensures that the modified sub-map still contains noise information from the noise residual map, guaranteeing that the subsequent tampering score is based on the noise residual map.

[0025] Optionally, determining the first tamper score of the baseline subgraph through a tamper scoring model includes:

[0026] The feature sub-map of the baseline sub-map is extracted through the feature extraction layer;

[0027] The tampering probability vector of the feature sub-image is obtained through the feature classification layer; the tampering probability vector is used to characterize the probability of each tampering level.

[0028] The first tampering score is determined based on the tampering probability vector and the tampering levels.

[0029] First, extract the feature sub-map of the baseline sub-map. This allows the feature classification layer to derive a tampering probability vector based on the feature sub-map. The tampering probability vector is used to represent the probability of each tampering level. In this way, the accuracy of deriving the first tampering score from the baseline sub-map can be improved.

[0030] Optionally, the tampered region of the target image is determined by using the N sub-images with the highest tampering scores corresponding to the N reference sub-images, including:

[0031] Cluster the N tampered scores;

[0032] Determine the anomaly threshold based on the clustering results;

[0033] The regions in the noise residual map where the M sub-images with tampering scores greater than the abnormal threshold are identified as the tampered regions of the target image.

[0034] Since not all of the N sub-images with the highest tampering scores necessarily contain tampering marks, but could also be normal noise, clustering is performed on the N sub-images with the highest tampering scores. The clustering results are then used to further determine the tampered areas of the target image. This improves the recognition rate of small-sized tampering marks.

[0035] Optionally, the regions in the noise residual map of the M sub-images with tampering scores greater than the anomaly threshold are determined as the tampered regions of the target image, including:

[0036] Identify the overlapping region between any two subgraphs in the noise residual map from the M subgraphs; if the overlapping region is greater than a set ratio, it is determined to be the same tampered region; if the overlapping region is not greater than the set ratio, it is determined to be different tampered regions.

[0037] Since multiple tampered areas may contain the same tampering trace, a limit is set on the proportion of overlapping areas between them. If the overlap is too large, it is considered to contain the same tampering trace and is thus identified as the same tampered area. This minimizes the possibility of the same tampering trace being repeatedly defined in the obtained tampered areas, improving the accuracy of tampered area detection.

[0038] Optionally, the tampering scoring model is obtained by training and learning based on the tampering scores of each sub-image of the sample image and the clustering results of the tampering scores of each sub-image of the sample image.

[0039] By continuously training and learning the tampering scoring model using the tampering scores and clustering results of each sub-image of the sample image, the accuracy of the tampering scoring results of the tampering scoring model can be improved.

[0040] Secondly, embodiments of the present invention also provide an image tampering detection device, comprising:

[0041] Processing unit, used for:

[0042] The target image is subjected to noise extraction processing to obtain a noise residual map;

[0043] The noise residual map is divided into N baseline sub-maps;

[0044] Determine the unit, used for:

[0045] For any baseline subgraph, the subgraph with the largest tampering score is determined by the tampering scoring model; the subgraph with the largest tampering score is one of the baseline subgraph and the modified subgraph of the baseline subgraph; the modified subgraph is the subgraph obtained by expanding or shrinking the region of the baseline subgraph in the noise residual map;

[0046] The tampered area of ​​the target image is determined by identifying the N sub-images with the highest tampering scores corresponding to the N reference sub-images.

[0047] Thirdly, embodiments of the present invention also provide a computing device, comprising:

[0048] Memory, used to store computer programs;

[0049] The processor is configured to invoke a computer program stored in the memory and execute the image tampering detection method listed in any of the above methods according to the obtained program.

[0050] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer-executable program, the computer-executable program being used to cause a computer to perform the image tampering detection method listed in any of the above embodiments. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 An image tampering detection method provided in this embodiment of the invention;

[0053] Figure 2 A schematic diagram of a terminal interface for acquiring a target image to be detected, provided in an embodiment of the present invention;

[0054] Figure 3A A schematic diagram illustrating a sampling reference sub-graph process provided in an embodiment of the present invention;

[0055] Figure 3B A schematic diagram illustrating a sampling reference sub-graph process provided in an embodiment of the present invention;

[0056] Figure 3C A schematic diagram illustrating a sampling reference sub-graph process provided in an embodiment of the present invention;

[0057] Figure 3D A schematic diagram illustrating a sampling reference sub-graph process provided in an embodiment of the present invention;

[0058] Figure 4 This is a flowchart illustrating a method for determining the subgraph with the highest tampered score, provided by an embodiment of the present invention.

[0059] Figure 5AThis is a schematic diagram illustrating a modification of a reference sub-graph by enlarging or shrinking it, provided as an embodiment of the present invention.

[0060] Figure 5B This is a schematic diagram illustrating a modification of a reference sub-graph by enlarging or shrinking it, provided as an embodiment of the present invention.

[0061] Figure 5C This is a schematic diagram of determining the subgraph with the largest tampered score according to an embodiment of the present invention;

[0062] Figure 6 A schematic diagram illustrating the determination of an overlapping region provided in an embodiment of the present invention;

[0063] Figure 7 An image tampering detection method provided in this embodiment of the invention;

[0064] Figure 8 This is a schematic diagram of the structure of an image tampering detection device provided in an embodiment of the present invention. Detailed Implementation

[0065] To make the objectives, implementation methods and advantages of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0066] Based on the exemplary embodiments described in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the appended claims. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete implementation on its own.

[0067] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0068] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities and do not necessarily imply a specific order or sequence, unless otherwise indicated. It should be understood that such terms can be used interchangeably where appropriate, for example, to implement the application in a sequence other than those given in the embodiments illustrated or described herein.

[0069] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that includes a series of components is not necessarily limited to those that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0070] The image tampering detection method provided in this embodiment of the invention continuously changes the region of the sub-image, making the region of the sub-image adapt to small-sized tampering traces as much as possible, thereby accurately determining the tampered area of ​​the target image.

[0071] The image tampering detection method provided in this embodiment of the invention can be executed by a server or a terminal device. The terminal device can be a computer such as a laptop or desktop computer, or a mobile terminal device such as a mobile phone or tablet. The server can be a server for a specific business (such as a transaction or financial business) or a server requiring risk control (such as risk control based on the detection, identification, or comparison of user privacy data such as facial images, fingerprints, or iris scans). Specifically, the server can be a payment service server, or a server related to finance or instant messaging. This embodiment of the invention does not impose any limitations on this.

[0072] Figure 1 An exemplary embodiment of the image tampering detection method provided by the present invention is shown. It includes the following steps:

[0073] Step 101: Perform noise extraction processing on the target image to obtain a noise residual map.

[0074] The target image can be any image, such as a person's facial image, fingerprint image, iris image, or an image of a building, or an image of a certificate with a signature, number, or seal, etc. The specific image can be set according to the actual situation, and this specification does not limit it in the embodiments.

[0075] The target image to be detected can be obtained in a variety of different ways, such as... Figure 2 As shown, an application for uploading images can be pre-configured, and an image upload entry can be set within that application. When a user needs to upload a target image to be detected, they can launch the application and locate the image upload entry. The target image can then be uploaded through this entry. After the upload is complete, the user's terminal device can acquire the target image and generate an image detection request based on it. This request is then sent to the server, allowing the server to obtain the target image to be detected.

[0076] Alternatively, users can upload the target images to be detected in batches to the server for image detection. The specific processing of the target images to be detected can be set according to the actual situation, and this specification does not limit this.

[0077] Since the noise fluctuations at the boundary of the tampered part are more drastic than those at the untampered part if the target image is tampered with, the noise residual map of the target image can be extracted and used for subsequent analysis.

[0078] The noise residual map can be image noise other than the principal components such as the displayable image content. For example, it can include normal noise introduced during image acquisition or image signal transmission and / or more severe abnormal noise introduced due to image tampering.

[0079] Specifically, noise extraction processing of the target image can be performed in a variety of different ways. For example, a corresponding filter can be set to separate the principal components and image noise contained in the target image, thereby obtaining the image noise of the target image, and then obtaining the noise residual map corresponding to the target image.

[0080] In practical applications, besides determining the noise residual of the target image using the methods described above, there are other ways to do so. For example, multiple different sample images can be pre-acquired, and an appropriate algorithm (such as ResNet (Residual Neural Network), network algorithm, etc.) can be pre-selected to construct the architecture of a model for extracting image noise. Then, the acquired sample images can be used to train the model, resulting in a trained model. This trained model can then be used to extract image noise from the target image, yielding the corresponding noise residual map.

[0081] The image can also be filtered using a noise filtering kernel to extract its noise residual map. Many types of noise filtering kernels can be used; one example is provided below:

[0082]

[0083] By performing a weighted operation on the pixels in an image with a noise filter kernel, a noise residual map of the image can be obtained. For example, if an ID card image of size 600*400 is input, after performing a weighted operation with the noise filter kernel τ, a noise residual map of size 600*400 can be obtained.

[0084] Besides determining the noise residual corresponding to the target image through the methods described above, it can also be achieved through various other methods. The specific method can be set according to the actual situation, and the embodiments of the present invention do not limit this.

[0085] Step 102: Divide the noise residual map into N baseline sub-maps.

[0086] There are multiple ways to obtain N subgraphs. Two methods are provided below.

[0087] Method 1

[0088] A sliding window is used to sample the noise residual map at a set step size to obtain N reference submaps.

[0089] The sliding window has a size of L*L, where L can be set to any value. Since numbers (such as property area on a real estate certificate, ID number, etc.) and text (signature, company name, etc.) are the easiest targets for tampering with document images, the font size in the document can be set as the initial length of the sliding window, for example, 20px. This embodiment of the invention does not impose any limitations on this.

[0090] To facilitate understanding, the process of sampling the baseline subplot will be explained below using a 6*4 noise residual plot as an example. Figure 3A As shown, the sliding window size is 2*2. If sliding sampling is performed with a step size of 1, 5*3 reference sub-images will be obtained; if sliding sampling is performed with a step size of 2, 3*2 reference sub-images will be obtained. It is evident that for the same noise residual image, different sliding step sizes result in different numbers of reference sub-images. The smaller the step size, the more reference sub-images are obtained, leading to higher accuracy in subsequently determining the tampered area of ​​the target image. This embodiment of the invention does not impose limitations on this.

[0091] If the step size is the same as the size of the sliding window, the number of subgraphs obtained can be expressed by the following formula:

[0092]

[0093] Where m is the horizontal dimension of the noise residual map, and n is the vertical dimension. Rounding up is necessary because if the sliding window reaches the boundary of the noise residual map and the size of the sampled reference submap is insufficient for the sliding window's size, zero-padding will be applied to the boundary region. For example... Figure 3B As shown, the noise residual map is 5*4, and the sliding window size is 2*2. If sliding sampling is performed with a step size of 2, at the boundary of the noise residual map, because the sampled reference sub-map is not 2*2, zero-padding is performed on the boundary region to obtain the reference sub-map A at the boundary. Finally, 3*2 reference sub-maps will be obtained.

[0094] By using sliding window sampling, multiple reference sub-images can be obtained based on the step size. The smaller the step size, the more reference sub-images are obtained, thus allowing for control over the accuracy of the final detection of tampered areas.

[0095] Method 2

[0096] The noise residual map is directly divided into N baseline sub-maps. For example, as Figure 3C As shown, the 6*4 noise residual map is directly divided into 3*2 reference sub-maps.

[0097] It is worth noting that the methods for segmenting the noise residual map are not limited to the two methods mentioned above. Various methods for segmenting the noise residual map can be conceived by those skilled in the art, and all should fall within the protection scope of the embodiments of the present invention. Regardless of the method used to segment the noise residual map, the sizes of the resulting N reference sub-maps can be equal or unequal; the reference sub-maps can overlap or not overlap; the segmented sub-maps can be rectangles or other polygons or circles, and the embodiments of the present invention do not impose any restrictions on this. Figure 3D One possible segmentation method is shown, in Figure 3D In the diagram, the reference subgraphs A, B, and C have different sizes and overlap in some areas.

[0098] Step 103: For any baseline subgraph, determine the subgraph with the largest tampering score using the tampering scoring model. The subgraph with the largest tampering score is one of the baseline subgraph and the modified subgraph of the baseline subgraph; the modified subgraph is the subgraph obtained by expanding or shrinking the region of the baseline subgraph in the noise residual map.

[0099] In this step, various changes, such as enlarging or shrinking, are needed to the regions of each baseline submap in order to obtain the submap with the highest tampering score. Two methods for determining the submap with the highest tampering score are provided below.

[0100] Method 1

[0101] Figure 4 This paper illustrates a method for determining the subgraph with the largest tampered score, comprising the following steps:

[0102] Step 401: Input the benchmark subgraph into the tampering scoring model, determine the first tampering score of the benchmark subgraph through the tampering scoring model, and use the first tampering score as the tampering benchmark score.

[0103] Step 402: Determine the changed submap of the baseline submap from the noise residual map according to the set regional change rules.

[0104] When modifying the baseline subgraph, various regional modification rules can be followed. For example, you can first expand it until the second tampered score is no greater than the tampered baseline score, and then shrink it until the second tampered score is no greater than the tampered baseline score; or you can first shrink it until the second tampered score is no greater than the tampered baseline score, and then expand it until the second tampered score is no greater than the tampered baseline score; or you can alternate between expanding and shrinking until the second tampered score is no greater than the tampered baseline score.

[0105] When scaling up or down a reference sub-image, the area of ​​the reference sub-image can be scaled up or down proportionally in the noise residual image, using the center of the reference sub-image as the center. For example, the top, bottom, left, and right parts of the reference sub-image can be scaled up or down by one pixel. Figure 5A As shown, the modified sub-map A1 is obtained after the baseline sub-map A is enlarged proportionally once; alternatively, the area of ​​the baseline sub-map can be enlarged or reduced non-proportionally in the noise residual map, using the center position of the baseline sub-map as the center, such as... Figure 5B As shown, the base sub-image A is expanded upward by 2 pixels, downward by 2 pixels, shrank to the left by 1 pixel, and shrank to the right by 1 pixel to obtain the modified sub-image A1.

[0106] Those skilled in the art can make various modifications to the method of obtaining the modified sub-graph and the rules for regional modification, and are not limited to the above-described examples of the embodiments of the present invention.

[0107] Step 403: Determine the second tamper score of the changed subgraph by tampering with the scoring model.

[0108] Step 404: If the second tampering score is greater than the tampering baseline score, then update the tampering baseline score to the second tampering score, return to step 402, until the second tampering score is no greater than the tampering baseline score.

[0109] Step 405: The subgraph corresponding to the tampered baseline score is determined as the subgraph with the largest tampered score.

[0110] The following is based on Figure 5C For example, we will explain the method for determining the subgraph with the highest tampered score.

[0111] Figure 5CThe image shows a 6*7 noise residual map with a small tampering mark 'a' (the tampering mark is not shown in the noise residual map, but in the target image; this is shown here to help the reader understand how the sub-map with the largest tampering score is determined). The region defined by the baseline sub-map A is shown in the figure. Inputting the baseline sub-map A into the tampering scoring model yields a first tampering score of 0.548, which is used as the baseline tampering score. Then, the baseline sub-map is enlarged to obtain the modified sub-map A1. The tampering scoring model obtains a second tampering score of 0.6 for the modified sub-map A1. Since the second tampering score is greater than the baseline tampering score, the second tampering score of 0.6 is used as the baseline tampering score. Continuing to modify the subgraph, we can choose to either expand or shrink it. If we choose to expand, we obtain subgraph A2. The tampering scoring model yields a second tampering score of 0.39 for subgraph A2. Since this second tampering score is less than the baseline tampering score, we stop expanding and begin shrinking the subgraph, resulting in subgraph A3. The tampering scoring model yields a second tampering score of 0.31 for subgraph A3. This second tampering score is also less than the baseline tampering score. Because the subgraph size is now at its minimum, we cannot shrink it further, so we stop shrinking. Thus, we obtain subgraph A1, which has the largest tampering score compared to the baseline subgraph A.

[0112] The above method determines the subgraph with the highest tampering score for any given baseline subgraph. By performing this method on each baseline subgraph, N baseline subgraphs will yield N subgraphs with the highest tampering scores.

[0113] By combining the tampering score of the tampering scoring model with adaptive changes to the region of the baseline sub-image, a sub-image with the highest tampering score can be determined for each baseline sub-image. In the sub-images obtained in this way, if there are tampering traces, the small-sized tampering traces will have the highest relative proportion in the sub-image, which can effectively prevent small-sized tampering traces from being missed.

[0114] Method 2

[0115] Besides modifying each baseline subgraph individually using the method described in Method 1, each baseline subgraph can also be modified simultaneously. For example, all subgraphs can be simultaneously expanded to obtain their respective tampering scores X1; expanded again to obtain their respective tampering scores X2; expanded again to obtain their respective tampering scores X3, until they reach the boundary of the noise residual map; then shrunk to obtain their respective tampering scores X1', shrunk again to obtain their respective tampering scores X2', until they cannot be shrunk further. In this way, multiple tampering scores will be obtained for each baseline subgraph. Among these multiple tampering scores, the subgraph corresponding to the highest tampering score is determined as the subgraph with the largest tampering score.

[0116] The above are merely examples, and the embodiments of the present invention do not impose any limitations on them.

[0117] In step 401, there are multiple ways to determine the first tampered score of the benchmark subgraph by tampering with the scoring model.

[0118] For example, a feature extraction layer can be set to extract feature sub-maps of a baseline sub-map, and then a feature classification layer can be used to obtain the tampering probability vector of the feature sub-map. The tampering probability vector is used to characterize the probability of each tampering level. Finally, the first tampering score is determined based on the tampering probability vector and each tampering level.

[0119] The above method is illustrated below with an example. Any reference sub-image z is input into the feature extraction layer to extract the tampering features of the reference sub-image. The feature extraction layer can be a convolutional layer + pooling layer + convolutional layer + pooling layer, or it can include convolutional layers + convolutional layers + pooling layers, etc. This embodiment of the invention does not limit the structure of the feature extraction layer. For example, if the size of the input reference sub-image is 20*20*1, after calculation with two 3*3*3 convolutional kernels and one 2*2 pooling kernel, the resulting feature sub-image is 8*8*9. Then, the feature sub-image is input into the feature classification layer to obtain the tampering probability vector βz = [0.01 0.02 0.06 0.11 0.19 0.2 0.2 0.16 0.07 0.03 0.01], where each element in the vector βz corresponds to the tampering level γ = [θ0 θ1 θ2 … θ9 θ 10 ] T =[0 0.1 0.2 … 0.9 1] T The probability of.

[0120] The first tampering score can be calculated using the following formula: σ z =β Z γ, z ∈ (0-Z). For example, in the example above, the first tampering score of the baseline subgraph z is σ. z =β Z γ = 0.548.

[0121] For example, a tampering scoring model may include a feature extraction layer and a feature classification layer, ultimately obtaining a tampering probability vector for each baseline sub-image. This tampering probability vector is the first tampering score of the baseline sub-image.

[0122] For example, the tampering scoring model includes a feature extraction layer and a feature classification layer, which ultimately obtains the tampering probability vector of each baseline sub-image. Then, the tampering probability vector is multiplied by the tampering grade by manual or computer calculation to obtain the first tampering score.

[0123] The above are merely examples, and the embodiments of the present invention do not limit the model structure for determining the first tampered score or the method for determining the first tampered score.

[0124] It is worth noting that in step 102, if the size of the sampled reference submap is insufficient for the sliding window size when the sliding window reaches the boundary of the noise residual map, zero-padding will be performed on the boundary region. Figure 3B As shown. Since half of the area of ​​the obtained reference sub-image A is obtained by padding with zeros, this half cannot show any signs of tampering. Therefore, when calculating the first tampering score, it should be given a corresponding regional weight. For example, in this case, the first tampering score obtained by the above method is 0.3. Then, the first tampering score can be divided by the ratio of non-zero-padded pixels to the total number of pixels in the reference sub-image, which is 1 / 2, to obtain the final first tampering score of 0.6.

[0125] Optionally, the feature classification layer is a feedforward neural network.

[0126] The method for determining the second tampered score is the same as that for determining the first tampered score, and will not be repeated here.

[0127] Step 104: Determine the tampered area of ​​the target image by using the N sub-images with the highest tampering scores corresponding to the N reference sub-images.

[0128] Since not all of the N sub-images with the highest tampering scores necessarily show signs of tampering, and may also be normal noise introduced during image acquisition or image signal transmission, the tampered areas of the target image can be further determined based on the N sub-images with the highest tampering scores.

[0129] There are many ways to determine the tampered area. The embodiments of the present invention provide the following methods.

[0130] Method 1

[0131] The tampered areas were identified using clustering methods.

[0132] Clustering is performed on N tampering scores to obtain multiple cluster centers. An anomaly threshold is determined based on these cluster centers. Then, the regions in the noise residual map of the M sub-images with tampering scores greater than the anomaly threshold are identified as the tampered regions of the target image. There are various ways to determine the anomaly threshold. For example, the cluster center with the largest value can be used as the anomaly threshold; the mean of multiple cluster centers can be used; or the median value between the largest and smallest cluster centers can be used. This embodiment of the invention does not limit the method used.

[0133] For example, after clustering N tampered ratings, two cluster centers are obtained with values ​​of 0.87 and 0.34. The median value between the two cluster centers, (0.87 + 0.34) / 2 = 0.605, is used as the anomaly threshold. If M tampered ratings are identified from the N tampered ratings that are greater than this anomaly threshold, then the regions in the noise residual map corresponding to these M tampered ratings are identified as the tampered regions of the target image.

[0134] When performing clustering, you can cluster either the tampering probability vector or the tampering score.

[0135] The cluster distance formula when using tampered scores for clustering is: Among them, Z i For any subgraph, c j For cluster centers, σ z Let σ be the tampering score for any subgraph, and σ0 be the initial cluster center.

[0136] The cluster distance formula when using tampered probability vectors for clustering is: Where P0 represents the probability values ​​in the tampering probability vector of the cluster centers, and γ t P is a constant. ti Let P be the probability value in each tampering probability vector of any subgraph. ti γ t and P0γ t The difference is the deviation distance between the probability of the subgraph and the cluster center under the t-classification feature.

[0137] Method 2

[0138] The N tampering scores are arranged in descending order. The sub-images corresponding to the M tampering scores that satisfy the first ratio are identified as the tampered regions of the target image. For example, the sub-images corresponding to the top 20% of the tampering scores are identified as the tampered regions of the target image.

[0139] The above are merely examples, and the embodiments of the present invention do not impose any limitations on them.

[0140] Based on the above steps, M sub-images are obtained. Since multiple sub-images may contain the same tampering trace, the overlapping area of ​​any two sub-images in the noise residual map is determined. If the overlapping area is greater than a set ratio, it is determined to be the same tampering area; if the overlapping area is not greater than the set ratio, it is determined to be different tampering areas.

[0141] Thus, by limiting the proportion of overlapping areas between different tampered areas, if the overlapping area is too large, it is considered to include the same tampering trace and is thus identified as the same tampered area. This minimizes the possibility that the obtained tampered areas will repeatedly identify the same tampering trace, improving the accuracy of tampered area detection.

[0142] For example, if the above steps identify four sub-images that may be tampered with, they can be represented in the format (x-coordinate of center position, y-coordinate of center position, size of sub-image) as A(250,180,7), B(450,310,11), C(253,178,6), and D(455,307,10). Figure 6 As shown, the overlapping areas between any two subgraphs in the four subgraphs are determined. If the overlapping area is greater than a set ratio of 75%, it is determined to be the same tampered area. For example, if the overlapping area between A and C is determined to be 80%, and the overlapping area between B and D is determined to be 90%, then they are determined to be the same tampered area. Then, duplicate tampered areas are removed, and tampered areas A and B, or tampered areas C and D, are retained.

[0143] One way to determine the overlapping region is to take the ratio of the intersection of the two subgraphs to their union as the overlapping region. For Figure 6 For A and C in the diagram, their intersection is the diagonal line portion, and their union is the shaded portion. Calculations determined that the overlapping area between them is greater than a predetermined ratio.

[0144] Optionally, embodiments of the present invention also provide a method for training the above-mentioned tampering rating model. The sample image is divided into N sub-images, and the N sub-images and their corresponding tampering ratings are input into the tampering rating model, so that the tampering rating model is trained based on these sub-images and their corresponding tampering ratings.

[0145] Optionally, the clustering results of the tampered scores can also be used as training material for the tampered score model. In method one of step 104, clustering is used to determine the tampered region. The number of cluster centers can be limited. If the number of cluster centers is greater than two, the parameters of the tampered score model are tuned until the number of cluster centers is two or one. This will determine the anomaly threshold more accurately and quickly.

[0146] By continuously training and learning the tampering scoring model using the tampering scores and clustering results of each sub-image of the sample image, the accuracy of the tampering scoring results of the tampering scoring model can be improved.

[0147] To better explain the embodiments of the present invention, the overall process of image tampering recognition described above will be explained below. For example... Figure 7 As shown, it includes the following steps:

[0148] Step 701: Obtain the target image.

[0149] Step 702: Perform noise extraction processing on the target image to obtain a noise residual map.

[0150] Step 703: Divide the noise residual map into N baseline sub-maps.

[0151] Step 704: For any baseline subgraph, determine the first tamper score through the tamper scoring model, and use the first tamper score as the tamper baseline score.

[0152] Step 705: Determine the changed submap of the baseline submap from the noise residual map according to the set regional change rules.

[0153] Step 706: Determine the second tampered score of the changed subgraph by tampering with the scoring model.

[0154] Step 707: Determine whether the second tampered score is greater than the tampered baseline score. If yes, proceed to step 708; otherwise, proceed to step 709.

[0155] Step 708: Update the tampering baseline score to the second tampering score.

[0156] Step 709: The subgraph corresponding to the tampered baseline score is determined as the subgraph with the largest tampered score.

[0157] Step 710: Cluster the N subgraphs with the largest tampering scores corresponding to the N baseline subgraphs to determine the M tampering regions.

[0158] Step 711: Determine whether the overlapping area of ​​any two subgraphs in the noise residual map is greater than a set ratio.

[0159] Step 712: Identify the different tampered areas.

[0160] Step 713: Identify the same tampered area.

[0161] Step 714: Output the tampered area.

[0162] The output of the tampered area can be the center coordinates and size of the tampered area, or the tampered area can be directly circled in the target image. This embodiment of the invention does not impose any limitations on this.

[0163] Because noise fluctuations are significant at the tampering boundaries in the target image, a noise residual map is obtained by extracting noise from the target image. This residual map can then be used to derive a tampering feature score, quantifying the degree of tampering. After dividing the noise residual map into N baseline sub-images, the regions of these sub-images are adaptively changed based on the tampering scoring model. This process ultimately identifies the N sub-images with the highest tampering scores. In these sub-images, if tampering traces are present, small-sized traces will have the highest relative proportion, effectively preventing them from being missed. However, not all of the N sub-images with the highest tampering scores necessarily contain tampering traces; some may simply be normal image noise. Therefore, the tampered regions in the target image are further determined based on these N sub-images. This improves the recognition rate of small-sized tampering traces.

[0164] Based on the same technological concept Figure 8 An exemplary embodiment of the present invention illustrates the structure of an image tampering detection device that can perform the image tampering detection process.

[0165] like Figure 8 As shown, the device specifically includes:

[0166] Processing unit 801 is used for:

[0167] The target image is subjected to noise extraction processing to obtain a noise residual map;

[0168] The noise residual map is divided into N baseline sub-maps;

[0169] Determine unit 802, used for:

[0170] For any baseline subgraph, the subgraph with the largest tampering score is determined by the tampering scoring model; the subgraph with the largest tampering score is one of the baseline subgraph and the modified subgraph of the baseline subgraph; the modified subgraph is the subgraph obtained by expanding or shrinking the region of the baseline subgraph in the noise residual map;

[0171] The tampered area of ​​the target image is determined by identifying the N sub-images with the highest tampering scores corresponding to the N reference sub-images.

[0172] Based on the same technical concept, embodiments of the present invention also provide a computing device, including:

[0173] Memory, used to store computer programs;

[0174] The processor is used to call a computer program stored in memory and execute the image tampering detection method listed in any of the above methods according to the obtained program.

[0175] Based on the same technical concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer-executable program, which is used to cause a computer to perform the image tampering detection method listed in any of the above methods.

[0176] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0177] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0178] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0179] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0180] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting image tampering, characterized in that, include: The target image is subjected to noise extraction processing to obtain a noise residual map; The noise residual map is divided into N baseline sub-maps; For any baseline subgraph, the subgraph with the largest tampering score is determined by the tampering scoring model; the subgraph with the largest tampering score is one of the baseline subgraph and its modified subgraph; the modified subgraph is obtained by expanding or shrinking the region of the baseline subgraph proportionally or non-proportionally, with the center position of the baseline subgraph as the center in the noise residual map; wherein, the tampering score is determined based on the tampering probability vector of the feature subgraph of the baseline subgraph and each tampering level; Cluster the N tampered scores; determine an anomaly threshold based on the clustering results; identify the M sub-images with tampered scores greater than the anomaly threshold as the tampered regions of the target image in the noise residual map; wherein the number of cluster centers obtained after clustering the N tampered scores must meet a preset requirement.

2. The method as described in claim 1, characterized in that, The subgraph with the largest score manipulation was identified by modifying the scoring model, including: The first tampered score of the benchmark subgraph is determined by the tampered scoring model, and the first tampered score is used as the tampered benchmark score; According to the set regional change rules, the changed submap of the baseline submap is determined from the noise residual map; The second tampering score of the altered subgraph is determined using the tampering scoring model; If the second tampering score is greater than the tampering baseline score, then the tampering baseline score is updated to the second tampering score, and the process returns to determine the changed sub-map of the baseline sub-map from the noise residual map according to the set regional change rules, until the second tampering score is no greater than the tampering baseline score; The subgraph corresponding to the tampered benchmark score is determined as the subgraph with the largest tampered score.

3. The method as described in claim 2, characterized in that, The noise residual map is divided into N baseline sub-maps, including: A sliding window is used to sample the noise residual map at a set step size to obtain N reference submaps; According to the established regional change rules, the changed submap of the baseline submap is determined from the noise residual map, including: According to the set regional change rules, the region of the reference sub-map is sampled in the noise residual map by expanding or shrinking the center position of the reference sub-map to obtain the changed sub-map of the reference sub-map.

4. The method as described in claim 2, characterized in that, Determining the first tampered score of the baseline subgraph using a tampered scoring model includes: The feature sub-map of the baseline sub-map is extracted through the feature extraction layer; The tampering probability vector of the feature sub-image is obtained through the feature classification layer; the tampering probability vector is used to characterize the probability of each tampering level. The first tampering score is determined based on the tampering probability vector and the tampering levels.

5. The method as described in claim 1, characterized in that, The regions in the noise residual map of the M sub-images whose tampering scores are greater than the anomaly threshold are identified as the tampered regions of the target image, including: Identify the overlapping region between any two subgraphs in the noise residual map from the M subgraphs; if the overlapping region is greater than a set ratio, it is determined to be the same tampered region; if the overlapping region is not greater than the set ratio, it is determined to be different tampered regions.

6. The method according to any one of claims 1-5, characterized in that, The tampering scoring model is trained and learned based on the tampering scores of each sub-image of the sample image and the clustering results of the tampering scores of each sub-image of the sample image.

7. An image tampering detection device, characterized in that, include: Processing unit, used for: The target image is subjected to noise extraction processing to obtain a noise residual map; The noise residual map is divided into N baseline sub-maps; Determine the unit, used for: For any baseline subgraph, the subgraph with the largest tampering score is determined by the tampering scoring model; the subgraph with the largest tampering score is one of the baseline subgraph and its modified subgraph; the modified subgraph is obtained by expanding or shrinking the region of the baseline subgraph proportionally or non-proportionally, with the center position of the baseline subgraph as the center in the noise residual map; wherein, the tampering score is determined based on the tampering probability vector of the feature subgraph of the baseline subgraph and each tampering level; Cluster the N tampered scores; determine an anomaly threshold based on the clustering results; identify the M sub-images with tampered scores greater than the anomaly threshold as the tampered regions of the target image in the noise residual map; wherein the number of cluster centers obtained after clustering the N tampered scores must meet a preset requirement.

8. A computing device, characterized in that, include: Memory, used to store computer programs; A processor is configured to invoke a computer program stored in the memory and execute the method according to any one of claims 1 to 6 in accordance with the obtained program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer-executable program for causing a computer to perform the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-6.

Images