An image copy-paste tampering detection method and system based on variable cluster spectrum clustering

Abstract

The application discloses a kind of variable cluster spectral clustering-based image copy-paste tampering detection method and system, comprising the following steps: S1, extracting key point feature information from the input image, construct key point set;S2, the key point set is layered feature matching, and matching point is obtained;S3, construct tendency vector, and utilize tendency vector to delete error matching point;S4, variable cluster spectral clustering is used to cluster matching point, and the matching point that meets similar transformation is divided into the same cluster, the variable cluster spectral clustering includes spectral clustering and dynamic adjustment to cluster;S5, estimate affine matrix by the matching point in cluster, and the tampering region of image is positioned according to affine matrix, and the positioning result of tampered region is output.This method compared with traditional technology, using tendency vector clustering, can dynamically adjust the number of clusters according to the number of tampering regions of the image to be detected, improve the accuracy of tampering detection.

Description

Technical Field

[0001] This invention relates to the field of digital image forensics, and more specifically, to a method and system for detecting image copy-paste tampering based on variable cluster spectrum clustering. Background Technology

[0002] The ever-advancing image processing technology has lowered the barrier to entry for digital image editing. With readily available image processing software, people can easily modify image content, and the altered images are often so realistic and seamless that the human eye cannot detect the tampering. Therefore, there is an urgent need for technology that can accurately detect whether images have been tampered with. This has led to the rapid development of digital image forensics technology.

[0003] Image region copy-paste manipulation is a common method of image tampering. This method involves copying a region of an image once or multiple times and pasting it to other locations within the same image. It may even involve copying and pasting multiple regions of the image simultaneously to alter its appearance. To conceal the tampering and increase the difficulty of detection, the tampered region is typically subjected to post-processing such as scaling, rotation, compression, noise addition, and lighting adjustments.

[0004] Currently, mainstream image region copy-paste tampering detection techniques are mainly divided into two types: block-based and keypoint-based detection techniques. Block-based techniques divide the image into many small blocks and detect whether copy-paste tampering has occurred by extracting and comparing the feature information of the blocks. Because block-based methods divide the entire image into many small blocks, the algorithm's running efficiency is too low. As a result, researchers have gradually focused their attention on keypoint-based detection techniques.

[0005] The basic idea of ​​keypoint-based image tampering detection methods is to extract keypoints from an image, obtain features describing these keypoints, and determine whether the image has been tampered with by detecting whether there are matching features. Lyu et al. proposed a typical keypoint-based detection algorithm. This method constructs Delaunay triangles from the extracted keypoints for matching, and iteratively expands the resulting matching Delaunay triangles outwards to ensure the triangles cover the entire tampered area. More lenient feature matching is then applied to the newly expanded areas. This method uses the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm to cluster the matching keypoints, handling multiple pairs of potentially tampered areas in the image. DBSCAN clusters points with a distance smaller than the set cluster radius into a single cluster and deletes clusters with too few points, achieving clustering and removing isolated points.

[0006] The drawback of the aforementioned prior art is that the DBSCAN used is sensitive to parameters, tends to cluster two clusters that are spatially close together, and tends to delete sparse but correct matching points as noise points, resulting in poor detection results.

[0007] Therefore, in light of the above requirements and the shortcomings of existing technologies, this application proposes an image copy-paste tampering detection method based on variable cluster spectrum clustering. Summary of the Invention

[0008] This invention provides an image copy-paste tampering detection method based on variable cluster spectrum clustering. During the detection process, it can effectively delete erroneous matches and dynamically adjust the number of clusters according to the number of tampered areas, achieving high detection accuracy for various types of copied-paste tampered images.

[0009] The primary objective of this invention is to solve the aforementioned technical problems. The technical solution of this invention is as follows:

[0010] The first aspect of this invention provides an image copy-paste tampering detection method based on variable cluster spectrum clustering, the method comprising the following steps:

[0011] S1. Extract key point feature information from the input image and construct a key point set.

[0012] S2. Perform hierarchical feature matching on the key point set to obtain matching points.

[0013] S3. Construct a bias vector and use the bias vector to delete incorrect matching points.

[0014] S4. The matching points are clustered using a variable cluster spectral clustering method, and the matching points that satisfy the similarity transformation are divided into the same cluster. The variable cluster spectral clustering method includes spectral clustering and dynamic adjustment of the clusters.

[0015] S5. Estimate the affine matrix by matching points within the cluster, locate the tampered area of ​​the image based on the affine matrix, and output the location result of the tampered area.

[0016] Furthermore, the set of key points is represented as follows:

[0017] P = {p1, p2, ..., p} m}

[0018] in, x i ,y i For key point p i The coordinates, σ i ,θ i p i Scale and direction.

[0019] The descriptive subset of each key point in the key point set is represented as follows:

[0020] D = {des1,des2,...,des} m}

[0021] Among them, des i The descriptor is a 128-dimensional vector.

[0022] Furthermore, the specific process of step S2 is as follows: the key points in the key point set P are grouped according to the gray value of the pixel at their location. After the grouping is completed, the descriptors of the key points are used for matching within each group to obtain the matching points.

[0023] Furthermore, the specific process of matching keypoint descriptors within each group is as follows: starting from the first keypoint p1 in the group, calculate the Euclidean distance between the descriptors of this keypoint and all other keypoints, and sort the Euclidean distances in ascending order to obtain the distance set Dis = {d1, d2, ..., d...}. m}, where d i (i = 1, ..., m) is the Euclidean distance between the descriptor of keypoint p1 and the descriptors of all other keypoints.

[0024] The specific process for obtaining the matching point is as follows:

[0025] d j / d j+1 <T d ,j∈(2,...,m-2)

[0026] Among them, T d This represents a preset threshold used to determine whether a descriptor matches; after matching, it outputs {d1,...d...} j}, where {d1,...d j The key points corresponding to} are all matching points of p1. A pair of matching points is represented as (p,q), where p,q∈P.

[0027] Based on the above technical features, since the tampered area is generally copied and pasted from a part of the image, the gray values ​​of corresponding pixels between a pair of tampered areas should be the same or similar. The key points in the key point set P are grouped according to the gray value of the pixels at their locations, and then descriptor matching is used within the group, instead of matching directly within the entire P. This will greatly shorten the time required for the matching process.

[0028] Furthermore, the specific process for constructing the tendency vector is as follows:

[0029] S31. Randomly select a key point p at one end of the matching pair. i Select the two closest key points and their matching points to construct the affine matrix H. i The affine matrix H i Specifically:

[0030]

[0031] Among them, t 11 ,t 12 ,t 21 ,t 22 t represents the scaling and rotation factors. x ,t y This represents the translation coefficient.

[0032] S32. Repeat step S31 500 times to obtain 500 affine matrices {H1,...,H...} 500}

[0033] S33, Obtain matching pairs (p i ,q i The tendency vector of ), wherein the tendency vector is specifically:

[0034] PS i =(PS) i (1),PS i (2),...,PS i (500))

[0035]

[0036] Among them, if the matching pair (p i ,q i Two points within a certain error satisfy the affine matrix H. j The affine transformation of PS will then... i The value of (j) is marked as 1, otherwise it is marked as 0.

[0037] Based on the aforementioned technical characteristics, unlike other traditional methods that use keypoint coordinates for clustering, this method constructs trend vectors to cluster matching points, effectively addressing the problem of poor clustering results caused by uneven point distribution density. Clustering based on trend vectors ensures that matching pairs satisfying similarity transformations are grouped into a single cluster. Generally, different tampered regions exhibit different affine transformations; therefore, constructing trend vectors for clustering can distinguish different tampered regions within the same image.

[0038] Furthermore, the specific process for deleting incorrect matching points is as follows: calculate the element PS in each tendency vector. iThe proportion of the number of elements with a value of 1 in (j) (j=1,...,500) relative to the total number of elements in the tendency vector is determined. If this proportion is less than a preset value, the matching pair is classified as an incorrect matching pair and deleted. Otherwise, it is not classified as an incorrect matching pair and is retained. An incorrect matching pair refers to a matching pair that does not belong to the target region of the modification, or a singular matching pair within the target region of the modification.

[0039] Furthermore, in step S4, the spectral clustering includes the following steps:

[0040] A1. Based on the n pairs of matching pairs remaining after deleting incorrect matching pairs, construct an n×n dimensional similarity matrix S of the matching pairs. n×n Calculate the overlap J(A,B) between the two vectors; where the tendency vector of the matched pair is PS. i (i = 1, ..., n), the elements in the similarity matrix are S (u,v) =J(PS) u PS v ), u,v=1,...,n, This represents the Jaccard distance between two vectors.

[0041] A2. Based on the similarity matrix S n×n The adjacency matrix W is calculated using the Gaussian kernel function. n×n , where δ is a preset value.

[0042] A3. Calculate the degree matrix D based on the adjacency matrix. n×n .

[0043] A4. Calculate the Laplacian matrix L = DW and normalize it, where the standard Laplacian matrix is...

[0044] A5. Calculate the standard Laplace matrix. The eigenvectors corresponding to the k smallest eigenvalues ​​in the matrix are used to obtain matrix V. n×k and V n×k The row vectors are standardized row by row, resulting in an n×k eigenma matrix F. n×k Where k represents the preset number of clusters, and the specific process of row-wise standardization is: dividing the element of each row in the matrix by the modulus of the row vector.

[0045] A6. Use the K-means clustering algorithm to analyze the feature matrix F. n×k Clustering is performed to obtain k clusters {C1,...,C...} k}

[0046] Furthermore, in step S4, the dynamic adjustment of the clusters specifically involves estimating each cluster to obtain an affine matrix, including the following steps:

[0047] B1. Select any three matching pairs from a cluster, calculate an affine matrix, and repeat the process a set number of times to obtain a set number of affine matrices.

[0048] B2. Calculate the affine matrix H with minimum error using the error formula. j The affine matrix H with minimum error j The optimal affine matrix for this cluster; the error formula is specifically as follows:

[0049]

[0050] B3. Repeat steps B1 and B2 to obtain the optimal affine matrices for several clusters.

[0051] B4. The optimal affine matrices are compared pairwise to determine whether the ratio of the error values ​​calculated using the matching points within the cluster of the two affine matrices is greater than a preset value. The error value is calculated by the above error formula. If it is, it means that the difference between the two optimal affine matrices is too large and they do not meet the same affine transformation, so they are not merged. Otherwise, the clusters containing the two optimal affine matrices are merged.

[0052] Based on the aforementioned technical characteristics, since the number of tampered regions in a tampered image is unknown, conventional spectral clustering requires setting the number of clusters at the beginning of clustering. This contradicts reality, as setting too few or too many clusters will affect the accuracy of the estimated affine matrix, thus impacting the accuracy of the localization results. However, the variable cluster spectral clustering method can dynamically adjust the number of clusters based on the number of tampered regions.

[0053] Furthermore, in step S5, the image localization result is represented by a binary image, with the tampered area displayed in white and the remaining areas displayed in black. Step S5 includes the following steps:

[0054] S51, Computational Cluster C i The correlation coefficient for each pixel a in region S is calculated as follows:

[0055]

[0056] Where c(a) represents the correlation coefficient, Ω(a) is a 5×5 pixel neighborhood centered at pixel a, μ is the pixel within Ω(a), and I(μ) represents the gray value of pixel μ. V(μ) represents the average gray value of the pixel region Ω(a), and V(μ) represents the gray value of the corresponding pixel after the affine matrix transformation of pixel μ. The grayscale value of the pixel region Ω(a) is obtained after affine transformation.

[0057] S52. Determine whether the correlation coefficient is greater than the preset value. If so, mark pixel a and its corresponding affine transformed pixel as tampered pixel. Otherwise, proceed to the next step.

[0058] S53. Repeat steps S51 and S52 until the correlation coefficient of each pixel in S is calculated and it is determined whether the pixel is a tampered pixel. Perform morphological operations on the input image and output the localization result of the tampered region.

[0059] A second aspect of the present invention provides an image copy-paste tampering detection system based on variable cluster spectrum clustering, comprising: a feature extraction module, a hierarchical feature matching module, a tendency vector construction module, a variable cluster spectrum clustering module, and a tampering region localization module.

[0060] The feature extraction module extracts key point feature information from the input image and constructs a key point set; the hierarchical feature matching module performs hierarchical feature matching on the key point set to obtain matching points; the tendency vector construction module constructs a tendency vector and uses the tendency vector to delete erroneous matching points; the variable cluster spectrum clustering module clusters the matching points and divides the matching points that satisfy the similarity transformation into the same cluster; the tampered region localization module estimates the affine matrix through the matching points in the cluster, locates the tampered region of the image according to the affine matrix, and outputs the localization result of the tampered region.

[0061] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0062] This invention provides an image copy-paste tampering detection method based on variable cluster spectrum clustering. The method uses variable cluster spectrum clustering to cluster the tendency vector, and can dynamically adjust the number of clusters according to the number of tampered areas, thereby achieving high detection accuracy for various types of copied-paste tampered images. Attached Figure Description

[0063] Figure 1 This is a flowchart of an image copy-paste tampering detection method based on variable cluster spectrum clustering according to the present invention.

[0064] Figure 2 This is an example image where copy-paste tampering has occurred.

[0065] Figure 3 This is a schematic diagram showing the actual tampered areas marked in the example image.

[0066] Figure 4 This is a schematic diagram of a region copy-paste tampering detection of an image to be detected in one embodiment of the present invention, wherein the white area is the area where copy-paste tampering was detected.

[0067] Figure 5This is a schematic diagram illustrating the effect of variable cluster spectrum clustering in one embodiment of the present invention.

[0068] Figure 6 This is a comparison chart of the detection effects of the present invention and existing technologies.

[0069] Figure 7 This is an example of an image region copy-paste tampering detection process in one embodiment of the present invention.

[0070] Figure 8 This is a schematic diagram of an image copy-paste tampering detection system based on variable cluster spectrum clustering according to the present invention. Detailed Implementation

[0071] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0072] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0073] Example 1

[0074] like Figure 1 As shown, this invention provides an image copy-paste tampering detection method based on variable cluster spectrum clustering. This method includes the following steps:

[0075] S1. Extract key point feature information from the input image and construct a key point set.

[0076] It should be noted that the SIFT (Scale Invariant Feature Transform) features extracted from images in this invention are invariant to changes in rotation, scaling, and brightness, and also maintain a certain degree of stability against noise. Using these features, attacks that tamper with images, such as rotation, scaling, compression, and noise addition, can be addressed.

[0077] S2. Perform hierarchical feature matching on the key point set to obtain matching points.

[0078] S3. Construct a bias vector and use the bias vector to delete incorrect matching points.

[0079] It should be noted that, unlike other traditional methods that use keypoint coordinates for clustering, this invention clusters matching points by constructing trend vectors, which effectively solves the problem of poor clustering results caused by uneven point distribution density. Clustering based on trend vectors ensures that matching pairs satisfying the similarity transformation (an affine transformation from one point in a matching pair to another) are grouped into one cluster. Generally, different tampered regions have different affine transformations; therefore, constructing trend vectors for clustering can distinguish different tampered regions within the same image.

[0080] S4. The matching points are clustered using a variable cluster spectral clustering method, and the matching points that satisfy the similarity transformation are divided into the same cluster. The variable cluster spectral clustering method includes spectral clustering and dynamic adjustment of the clusters.

[0081] S5. Estimate the affine matrix by matching points within the cluster, locate the tampered area of ​​the image based on the affine matrix, and output the location result of the tampered area.

[0082] In one specific embodiment, the image to be detected received by the present invention is an image that has been copied and pasted, such as... Figure 2 As shown in the image, there are identical or similar areas, and the actual areas where copying and pasting occurred are as follows: Figure 3 As shown, after the present invention performs regional copy-paste tampering detection, the output is as follows: Figure 4 The altered area location results shown can be seen as follows: Figure 4 The white area in the middle and Figure 3 The white areas in the image are almost uniform, demonstrating the high-precision identification effect achieved by the technical solution of this invention in identifying tampered areas. At the same time, by using a black background and white as the background color for tampered areas, it is easy to see which areas in the image have been tampered with, thus improving the efficiency of detection and the accuracy of identification.

[0083] Furthermore, the set of key points is represented as follows:

[0084] P = {p1, p2, ..., p} m}

[0085] in, x i ,y i For key point p i The coordinates, σ i ,θ i p i Scale and direction.

[0086] The descriptive subset of each key point in the key point set is represented as follows:

[0087] D = {des1,des2,...,des}m}

[0088] Among them, des i The descriptor is a 128-dimensional vector.

[0089] Furthermore, the specific process of step S2 is as follows: the key points in the key point set P are grouped according to the gray value of the pixel at their location. After the grouping is completed, the descriptors of the key points are used for matching within each group to obtain the matching points.

[0090] Furthermore, the specific process of matching keypoint descriptors within each group is as follows: starting from the first keypoint p1 in the group, calculate the Euclidean distance between the descriptors of this keypoint and all other keypoints, and sort the Euclidean distances in ascending order to obtain the distance set Dis = {d1, d2, ..., d...}. m}, where d i (i = 1, ..., m) is the Euclidean distance between the descriptor of keypoint p1 and the descriptors of all other keypoints.

[0091] The specific process for obtaining the matching point is as follows:

[0092] d j / d j+1 <T d ,j∈(2,...,m-2)

[0093] Among them, T d In a specific embodiment, T represents a preset threshold. d =0.6, used to determine if the descriptors match; after matching, output {d1,...d j}, where {d1,...d j The key points corresponding to} are all matching points of p1. A pair of matching points is represented as (p,q), where p,q∈P.

[0094] It should be noted that since the tampered area is usually copied and pasted from a part of the image, the gray values ​​of corresponding pixels between a pair of tampered areas should be the same or similar. The key points in the key point set P are grouped according to the gray value of the pixels at their location, and then descriptor matching is used within the group, instead of matching directly within the entire P. This will greatly shorten the time required for the matching process.

[0095] In one specific embodiment, the keypoint set P is divided into 5 groups, where the grayscale value range of each group is 60, and there is an overlap of 10 grayscale values ​​between adjacent groups, that is:

[0096]

[0097] Among them, l i For key point p i The grayscale value of the pixel at the given location.

[0098] Furthermore, the specific process for constructing the tendency vector is as follows:

[0099] S31. Randomly select a key point p at one end of the matching pair. i Select the two closest key points and their matching points to construct the affine matrix H. i The affine matrix H i Specifically:

[0100]

[0101] Among them, t 11 ,t 12 ,t 21 ,t 22 t represents the scaling and rotation factors. x ,t y This represents the translation coefficient. The six unknown coefficients mentioned above can be calculated based on the coordinates of three pairs of matching points. For a pair of matching points p... i =(x i ,y i ,1),q i =(x′) i ,y′ i ,1), satisfying the following expression:

[0102] q i ≈p i H i .

[0103] S32. Repeat step S31 500 times to obtain 500 affine matrices {H1,...,H...} 500}

[0104] S33, Obtain matching pairs (p i ,q i The tendency vector of ), wherein the tendency vector is specifically:

[0105] PS i =(PS) i (1),PS i (2),...,PS i (500))

[0106]

[0107] Among them, if the matching pair (p i ,q i Two points within a certain error satisfy the affine matrix H. jThe affine transformation of PS will then... i The value of (j) is marked as 1, otherwise it is marked as 0.

[0108] It should be noted that each matching pair (p) i ,q i Each of them has a corresponding 500-dimensional tendency vector, whose constituent elements are 0 or 1.

[0109] Furthermore, before clustering the matching points, incorrect matches need to be removed. The specific process is as follows: calculate the element PS in each tendency vector. i The proportion of elements (j) (j=1,...,500) that are 1 out of the total number of elements in the tendency vector is determined. If this proportion is less than a preset value, the matching pair is classified as an incorrect match and deleted; otherwise, it is not classified as an incorrect match and is retained. An incorrect match refers to a match that does not belong to the target region being modified, or a singular match within the target region. In a specific embodiment, the preset value is 5%.

[0110] It should be noted that since the number of tampered regions in a tampered image is unknown, ordinary spectral clustering requires setting the number of clusters at the beginning of clustering, which contradicts reality. Setting too few or too many clusters will affect the accuracy of the estimated affine matrix, thus affecting the accuracy of the localization results. The variable cluster spectral clustering algorithm proposed in this invention overcomes the limitation of ordinary spectral clustering, which can only form a pre-set number of clusters, by dynamically adjusting the number of clusters based on the number of tampered regions. The clustering process of variable cluster spectral clustering mainly consists of two parts: spectral clustering and dynamic cluster adjustment. In the first part, the number of clusters k is set. Generally, a value of 5 can effectively handle common tampering situations. For cases with few matching pairs, the number of pre-set clusters can be appropriately reduced based on the number of matching points to obtain the best clustering effect.

[0111] Furthermore, in step S4, the spectral clustering includes the following steps:

[0112] A1. Based on the n pairs of matching pairs remaining after deleting incorrect matching pairs, construct an n×n dimensional similarity matrix S of the matching pairs. n×n Calculate the overlap J(A,B) between the two vectors; where the tendency vector of the matched pair is PS. i (i = 1, ..., n), the elements in the similarity matrix are S (u,v) =J(PS) u PS v ), u,v=1,...,n, This represents the Jaccard distance between two vectors.

[0113] A2. Based on the similarity matrix Sn×n The adjacency matrix W is calculated using the Gaussian kernel function. n×n , where δ is a preset value. In a specific embodiment, δ is set to 0.2.

[0114] A3. Calculate the degree matrix D based on the adjacency matrix. n×n The degree matrix is ​​a matrix whose off-diagonal elements are 0, and the value of each diagonal element in each row is the sum of the values ​​of the corresponding rows of the adjacent matrix.

[0115] A4. Calculate the Laplacian matrix L = DW and normalize it, where the standard Laplacian matrix is...

[0116] A5. Calculate the eigenvectors corresponding to the k smallest eigenvalues ​​in the standard Laplacian matrix L to obtain matrix V. n×k and V n×k The row vectors are standardized row by row, resulting in an n×k eigenma matrix F. n×k Where k represents the preset number of clusters, and the specific process of row-wise standardization is: dividing the element of each row in the matrix by the modulus of the row vector.

[0117] A6. Use the K-means clustering algorithm to analyze the feature matrix F. n×k Clustering is performed to obtain k clusters {C1,...,C...} k}

[0118] Furthermore, in step S4, the dynamic adjustment of the clusters specifically involves estimating the affine matrix for each cluster. It should be noted that since the number of tampered regions in an image is unknown in reality, adjustment is needed to obtain a number of clusters equal to the number of tampered regions. First, the RANSAN algorithm is executed for each cluster to estimate an affine matrix. This algorithm includes the following steps:

[0119] B1. Randomly select three matching pairs from a cluster, calculate an affine matrix, and repeat this process a set number of times to obtain a set number of affine matrices. In a specific embodiment, repeating this process 1000 times yields 1000 affine matrices.

[0120] B2. Calculate the affine matrix H with minimum error using the error formula. j The affine matrix H with minimum error j The optimal affine matrix for this cluster; the error formula is specifically as follows:

[0121]

[0122] B3. Repeat steps B1 and B2 to obtain the optimal affine matrices for several clusters.

[0123] B4. The optimal affine matrices are compared pairwise to determine whether the ratio of the error values ​​calculated using the matching points within the cluster of the two affine matrices is greater than a preset value. The error value is calculated by the above error formula. If it is, it means that the difference between the two optimal affine matrices is too large and they do not meet the same affine transformation, so they are not merged. Otherwise, the clusters containing the two optimal affine matrices are merged.

[0124] In a specific embodiment, the affine matrix of cluster C1 is calculated, where C1 = {(p i ,q i )|i=1,...,n},(p i ,q i ) are matching pairs within cluster C1. Randomly selecting 3 matching pairs from the cluster yields an affine matrix. Performing this step 1000 times results in 1000 affine matrices {H}. j |j=1,...,1000};

[0125] Next, we search among these 1000 affine matrices for the affine matrix H that minimizes the error formula. j This affine matrix is ​​the optimal affine matrix for the cluster, and the specific error formula is as follows:

[0126]

[0127] Suppose there are two clusters C1 = {(p 1,i ,q 1,i )|i=1,...,n1),C2={(p 2,i ,q 2,i )|i=1,...,n2), while (p 1,i ,q 1,i ),(p 2,i ,q 2,i Let n1 and n2 be the number of matching pairs within the two clusters mentioned above. To determine whether clusters C1 and C2 should be merged, the following calculation process is required:

[0128]

[0129] Where H1 is the optimal affine matrix calculated by C1 using RANSAC, and H2 is the optimal affine matrix calculated by C2 using RANSAC. If the value calculated by the above formula is greater than 2500, it indicates that H1 and H2 are too different, meaning that the matching pairs in the two clusters do not satisfy the same affine transformation and should not be merged; if the value is less than 2500, the two clusters will be merged. Through the dynamic adjustment process of the clusters, the final number of clusters will remain consistent with the number of tampered regions.

[0130] Furthermore, in step S5, the image localization result is represented by a binary image, with the tampered area displayed in white and the remaining areas displayed in black. Step S5 includes the following steps:

[0131] S51, Computational Cluster C i The correlation coefficient for each pixel a in region S is calculated as follows:

[0132]

[0133] Where c(a) represents the correlation coefficient, Ω(a) is a 5×5 pixel neighborhood centered at pixel a, μ is the pixel within Ω(a), and I(μ) represents the gray value of pixel μ. V(μ) represents the average gray value of the pixel region Ω(a), and V(μ) represents the gray value of the corresponding pixel after the affine matrix transformation of pixel μ. The grayscale value of the pixel region Ω(a) is obtained after affine transformation.

[0134] S52. Determine whether the correlation coefficient is greater than a preset value. If so, mark pixel a and its corresponding affine transformed pixel as tampered pixels; otherwise, proceed to the next step. In a specific embodiment, the preset value is 0.6.

[0135] S53. Repeat steps S51 and S52 until the correlation coefficient of each pixel in S is calculated and it is determined whether the pixel is a tampered pixel. Perform morphological operations on the input image and output the localization result of the tampered region.

[0136] In a specific embodiment, the image copy-paste tampering detection method based on variable cluster spectrum clustering proposed according to the present invention achieves the following effect on the image to be detected: Figure 5 As shown, (a) is the tampered image; (b) is the initial clustering effect; (c) is the clustering effect after deleting incorrect matches; and (d) is the effect after dynamically adjusting the number of clusters. It can be seen that in the initial clustering effect image, besides the clearly regular matching pairs, there are also some random incorrect matching pairs. In the clustering effect image after deleting incorrect matches, the original tampered image only has 3 tampered regions but is divided into 5 clusters. After dynamically adjusting the number of clusters, the output result only retains three clusters, corresponding to the number of tampered regions.

[0137] Existing technologies tend to cluster matching pairs that are spatially close but belong to different tampered regions together, and they also tend to delete correct but sparsely distributed matching pairs, leading to inaccurate tampering detection results. The detection method employed in this invention effectively removes erroneous matches. By dynamically adjusting the number of clusters to maintain consistency with the number of tampered regions, it effectively improves the accuracy of the detection results.

[0138] The comparison chart of the detection effects of the present invention and the prior art is shown below. Figure 6 As shown, (a) is the clustering effect diagram of the prior art. Compared with the clustering effect diagram (b) of the present invention, some sparsely distributed matching pairs in (a) are deleted. The reason is that DBSCAN clusters sparsely distributed matching pairs into separate clusters. The number of pairs contained in these clusters is insufficient to determine an affine matrix, so they are treated as incorrect matching pairs and deleted. The location result in (c) is missing a part compared with the actual tampered area. The reason is that in the clustering method used in the prior art, the key point matching pairs in this part are sparsely distributed and are deleted as incorrect matching points, thus reducing the accuracy of the detection result. (d) is the detection result of the present method. Combined with the actual tampered area (e), the detection accuracy of the present method is high. (f) is the clustering effect diagram of the prior art. Compared with the clustering effect (g) of the present invention, (f) has one less cluster after clustering. The reason is that the clustering method used in the prior art clusters points that are too close together (points with a relatively dense distribution) together, as shown by the green lines connecting the parts in (f). (h) shows the localization results of existing technologies, which are significantly missing compared to the detection results (i) of our method. This is because the method should have divided the data into two clusters, each with an estimated affine matrix, but the result was divided into only one cluster, leading to inaccurate affine matrix estimation and affecting the localization results. Considering the actual tampered area (j), our method demonstrates higher detection accuracy.

[0139] Example 2

[0140] Based on the above embodiment 1, combined with Figure 7 This embodiment describes in detail the specific steps of the tampered region identification process of the present invention using a specific image to be detected.

[0141] In a specific embodiment, such as Figure 7 As shown in the figure, (a) represents the tampered image; (b) represents the feature extraction process; (c) represents the feature matching result; (d) represents the process of deleting incorrect matching pairs; (e) represents the result of variable cluster spectrum clustering; (f) represents the pixel calculation range; (g) represents the initial localization result; (h) represents the final localization result; and (i) represents the actual tampered area. The overall process includes the following steps:

[0142] ① Feature extraction: First, SIFT features are extracted from the image. The extraction process is as follows:

[0143] First, a scale space is established and a Gaussian difference image is obtained to achieve edge and corner detection. The scale space L(x,y,σ) of the image at different scales is obtained by convolving the original image I(x,y) with a Gaussian kernel G(x,y,σ):

[0144] L(x,y,σ)=I(x,y)*G(x,y,σ)

[0145]

[0146] Where σ is the scale parameter, the larger the value, the smoother the resulting image. Next, we calculate the Gaussian difference image in scale space, which is the difference between adjacent scale spaces:

[0147] D(x,y,σ)=L(x,y,σ)-L(x,y,kσ)

[0148] Next, key point localization is performed, which means comparing the point to be detected with its nearest neighbor. If the point has a maximum or minimum value, it is marked as a feature point. Neighboring points are 26 pixels, representing the 8 surrounding points in the same image layer and the 9 nearest points in the layers above and below.

[0149] Then, the orientation of the key points is determined. The orientation of the key points is obtained using the gradient histogram statistical method. The gradient magnitude and orientation of the pixels in the neighborhood centered on the key point are statistically analyzed. The magnitude m(x,y) of the gradient of each point L(x,y) and its orientation are shown in the following formula:

[0150]

[0151]

[0152] For the generated gradient direction histogram, the direction represented by its peak is set as the direction of the key point. This peak is called the main peak, and the direction is called the main direction.

[0153] After determining the location and orientation of the keypoints, it is necessary to calculate the SIFT feature descriptor for each point. The image patch surrounding the keypoint is divided into several sub-regions. The gradient magnitude and direction within each sub-region are calculated to obtain a feature vector. Finally, these feature vectors are combined to form a SIFT descriptor for the keypoint. Specifically, the image patch is divided into 4x4 sub-blocks, and each sub-block is further divided into 4 sub-regions. The gradient magnitude and direction of pixels within each sub-region are calculated. The direction is divided into 8 direction segments, and the gradient magnitudes of each direction segment are summed to obtain an 8-dimensional feature vector. Finally, the feature vectors of all 16 sub-blocks are concatenated to obtain a 128-dimensional SIFT descriptor.

[0154] in Figure 7 In (b), the red dots represent key points extracted from the image.

[0155] ② Hierarchical Feature Matching: First, the extracted SIFT feature descriptors are divided into 5 groups based on the pixel grayscale values ​​of their corresponding keypoint locations. The matching process is performed within each of these 5 groups. The process of matching one descriptor with other descriptors starts with the descriptor with the closest Euclidean distance, and d is calculated. j / d j+1 For each ,j∈(2,...,u-2), the value of j increases sequentially from smallest to largest until the ratio is greater than 0.6, d j Previously, the corresponding descriptors were all matching descriptors. This allows similar key points to be matched together, where... Figure 7 In (c), the keypoint pairs shown by connecting lines are the matched keypoint pairs.

[0156] ③ Constructing the tendency vector: First, 500 affine matrices are randomly calculated. Then, the mapping relationship between each matching point pair and these 500 affine matrices is determined. If the matching point pair satisfies the mapping relationship of a certain affine matrix, it is represented as 1 in the tendency vector; otherwise, it is represented as 0. Therefore, each matching point pair can obtain a 500-dimensional tendency vector composed of 0s and 1s.

[0157] ④ Delete incorrect matches: Determine if a match is correct based on the number of 1s in the constituent elements of the tendency vector. If the number of 1s in the tendency vector of a match is less than 5%, then delete the match. Figure 7 (d) shows the results of removing incorrect matches.

[0158] ⑤ Variable Cluster Spectral Clustering: First, spectral clustering is used to cluster the tendency vectors of matching point pairs. The number of clusters needs to be determined beforehand; here, it's set to 5. Because the number of spectral clusters is initially set, if the actual number of tampered regions is less than 5 pairs, the number of clusters will be inconsistent with the number of tampered regions. Therefore, dynamic adjustment of the clusters is necessary. The ratio of the optimal affine matrix of two clusters to the error of the matching pair in one of the clusters is used to determine whether the two clusters belong to the same cluster. For example, to determine cluster C... A With cluster C B Whether merging is necessary depends first on calculating cluster C using the error formula. A The matching pair within the matrix H corresponds to its affine matrix H. A The error value ξ1 and cluster C A Matching pairs within and cluster C B The corresponding affine matrix H B The error value ξ2 indicates that the larger the error value, the lower the matching degree between the affine matrix and the cluster. If the cluster C A With cluster C BIf the two clusters are not of the same type, then the value of ξ2 will be much larger than that of ξ1. Therefore, the ratio of ξ2 / ξ1 is used to determine whether the two clusters should be merged. If the ratio is greater than 2500, it indicates that cluster C is not the right one. A With cluster C B Clusters that are not of the same type should not be merged; conversely, clusters that are compatible should be merged. Figure 7 (e) is the final clustering result.

[0159] ⑥ Estimate the affine matrix: Use the RANSAC algorithm to iterate multiple times to estimate the affine matrix of each cluster. The obtained affine matrix is ​​the affine matrix in the cluster that minimizes the error formula, which is the optimal affine matrix.

[0160] ⑦ Locating the tampered area: The localization process involves representing the tampered area using a binary image. First, the approximate location of the tampered area must be determined. The approximate range of the tampered area is obtained by using a circular region centered on each key point as the pixel calculation range. Figure 7 (f) shows the approximate extent of the tampered region. Mapping all pixels within this range using an affine matrix yields a mapped region. The pixel correlation coefficient is calculated using the grayscale values ​​of corresponding pixels within these two regions, with the coefficient ranging from [0,1]. A higher coefficient indicates greater similarity between the two pixels. If the pixel correlation coefficient exceeds 0.6, the two pixels are marked as tampered pixels and recorded as 1 in the located binary image. The result is shown below. Figure 7 As shown in (g). Finally, morphological operations are performed on the image to obtain the final localization result, as shown in (g). Figure 7 As shown in (h). Figure 7 (i) shows the actual tampered area of ​​the tampered image, and it can be seen that the positioning result of the present invention is very close to the actual tampered result.

[0161] Example 3

[0162] like Figure 8 As shown, the present invention also provides an image copy-paste tampering detection system based on variable cluster spectrum clustering, including: a feature extraction module, a hierarchical feature matching module, a tendency vector construction module, a variable cluster spectrum clustering module, and a tampering region localization module.

[0163] The feature extraction module extracts key point feature information from the input image and constructs a key point set; the hierarchical feature matching module performs hierarchical feature matching on the key point set to obtain matching points; the tendency vector construction module constructs a tendency vector and uses the tendency vector to delete erroneous matching points; the variable cluster spectrum clustering module clusters the matching points and divides the matching points that satisfy the similarity transformation into the same cluster; the tampered region localization module estimates the affine matrix through the matching points in the cluster, locates the tampered region of the image according to the affine matrix, and outputs the localization result of the tampered region.

[0164] In one specific embodiment, a computer device is also provided, including a memory and a processor. The memory includes an image copy-paste tampering detection program based on variable cluster spectrum clustering. When the image copy-paste tampering detection program based on variable cluster spectrum clustering is executed by the processor, it implements an image copy-paste tampering detection method based on variable cluster spectrum clustering.

[0165] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] Alternatively, if the above embodiments of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device.

[0167] The methods described in the various embodiments of the present invention may be performed in whole or in part. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0168] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. The icons depicting structural positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting image copy-paste tampering based on variable cluster spectrum clustering, characterized in that, Includes the following steps: S1. Extract key point feature information from the input image and construct a key point set; S2. Perform hierarchical feature matching on the key point set to obtain matching points; S3. Construct a tendency vector and use the tendency vector to delete incorrect matching points; S4. Clustering matching points using a variable cluster spectral clustering method, grouping matching points that satisfy the similarity transformation into the same cluster. This variable cluster spectral clustering method includes spectral clustering and dynamic adjustment of the clusters. Specifically, the dynamic adjustment of the clusters involves estimating the affine matrix for each cluster, including the following steps: B1. Select any three matching pairs from a cluster, calculate an affine matrix, and repeat the process a set number of times to obtain a set number of affine matrices. B2. Calculate the affine matrix using the error formula to obtain the affine matrix with the minimum error. The affine matrix with minimum error The optimal affine matrix for this cluster; the error formula is specifically as follows: ； B3. Repeat steps B1 and B2 to obtain the optimal affine matrices for several clusters; B4. Perform pairwise comparisons of the optimal affine matrices and determine whether the ratio of the error values ​​calculated using the matching points within the cluster of the two affine matrices is greater than a preset value. If so, it means that the difference between the two optimal affine matrices is too large and they do not meet the same affine transformation, so they are not merged. Otherwise, the clusters containing the two optimal affine matrices are merged. S5. Estimate the affine matrix by matching points within the cluster, locate the tampered area of ​​the image based on the affine matrix, and output the location result of the tampered area.

2. The image copy-paste tampering detection method based on variable cluster spectrum clustering according to claim 1, characterized in that, The set of key points is represented as follows: in, , Key point coordinates They are respectively Scale and direction; The descriptive subset of each key point in the key point set is represented as follows: in, The descriptor is a 128-dimensional vector.

3. The image copy-paste tampering detection method based on variable cluster spectrum clustering according to claim 1, characterized in that, The specific process of step S2 is as follows: Set the key points Key points within the area are grouped according to the grayscale value of the pixels at their location. After grouping, the descriptors of the key points are used for matching within each group to obtain matching points.

4. The image copy-paste tampering detection method based on variable cluster spectrum clustering according to claim 3, characterized in that, The specific process of matching using keypoint descriptors within each group is as follows: starting from the first keypoint within the group... First, calculate the Euclidean distance between the keypoint and the descriptors of all other keypoints, and then sort the Euclidean distances in ascending order to obtain the distance set. ,in This is the key point. The Euclidean distance between the descriptor and all other keypoint descriptors; The specific process for obtaining the matching point is as follows: in, This represents a preset threshold used to determine whether a descriptor matches; the output is the value after a match is found. ,in The corresponding key points are all The matching point, a pair of matching points is represented as ,in .

5. The image copy-paste tampering detection method based on variable cluster spectrum clustering according to claim 4, characterized in that, The specific process for constructing the tendency vector is as follows: S31. Randomly select a key point at one end of the matching pair. Select the two closest key points and their matching points to construct an affine matrix. The affine matrix Specifically: in, Indicates scaling and rotation factors. Indicates the translation coefficient; S32. Repeat step S31 500 times to obtain 500 affine matrices. ; S33. Obtain matching pairs The tendency vector, specifically: Among them, if the matching pair Two points within a certain error satisfy the affine matrix The affine transformation will then The value is marked as 1, otherwise it is marked as 0.

6. The image copy-paste tampering detection method based on variable cluster spectrum clustering according to claim 5, characterized in that, The specific process for deleting incorrect matching points is as follows: Calculate the elements in each tendency vector. The proportion of the number of elements with the value 1 to the total number of elements in the tendency vector is used to determine if the proportion is less than a preset value. If it is, the matching pair is determined to be an incorrect matching pair and deleted. Otherwise, it is not determined to be an incorrect matching pair and is retained.

7. The image copy-paste tampering detection method based on variable cluster spectrum clustering according to claim 6, characterized in that, In step S4, the spectral clustering includes the following steps: A1. Based on the remaining pairs after deleting incorrect matches. For matching pairs, construct the matching pairs 3D similarity matrix Calculate the overlap between two vectors ; where the tendency vector of the matching pair is The elements in the similarity matrix are , , Represents the Jaccard distance between two vectors; A2. Based on the similarity matrix Calculate the adjacency matrix using the Gaussian kernel function. ,in This is the default value; A3. Calculate the degree matrix based on the adjacency matrix. ; A4. Calculate the Laplacian matrix. And it is standardized, where the standard Laplace matrix ; A5. Calculate the standard Laplace matrix. middle The eigenvectors corresponding to the smallest eigenvalues ​​yield the matrix. and will The row vectors are standardized row by row, and the final result is... Feature matrix ;in, The number of clusters is indicated by the row-wise normalization process, which is to divide the element of each row in the matrix by the modulus of the row vector. A6. Use the K-means clustering algorithm to analyze the feature matrix. Clustering is performed to obtain Cluster .

8. The image copy-paste tampering detection method based on variable cluster spectrum clustering according to claim 1, characterized in that, In step S5, the image localization result is represented by a binary image, with the tampered area displayed in white and the remaining areas displayed in black. Step S5 includes the following steps: S51, Computational Cluster Corresponding area Each pixel The correlation coefficient, and the calculation process of the correlation coefficient is as follows: in, Represents the correlation coefficient. It is based on pixels Centered Pixel neighborhood yes pixels within, represent The grayscale value of a pixel, represent The average gray level of a pixel region. represent The grayscale value of a pixel after undergoing an affine matrix transformation. represent The average gray value of a pixel region obtained after undergoing an affine transformation. S52. Determine if the correlation coefficient is greater than a preset value. If so, move the pixel... If the corresponding affine transformed pixels are marked as tampered pixels, otherwise proceed to the next step; S53. Repeat steps S51 and S52 until... For each pixel in the image, the correlation coefficient is calculated and it is determined whether the pixel is a tampered pixel. Morphological operations are performed on the input image, and the location result of the tampered region is output.

9. An image copy-paste tampering detection system based on variable cluster spectrum clustering, comprising: a feature extraction module, a hierarchical feature matching module, a tendency vector construction module, a variable cluster spectrum clustering module, and a tampering region localization module; The feature extraction module extracts key point feature information from the input image and constructs a key point set; the hierarchical feature matching module performs hierarchical feature matching on the key point set to obtain matching points; the tendency vector construction module constructs a tendency vector and uses the tendency vector to delete erroneous matching points; the variable cluster spectral clustering module clusters the matching points, dividing matching points that satisfy similarity transformation into the same cluster. The variable cluster spectral clustering method includes spectral clustering and dynamic adjustment of clusters; the dynamic adjustment of clusters specifically involves estimating each cluster to obtain an affine matrix, including the following steps: B1. Select any three matching pairs from a cluster, calculate an affine matrix, and repeat the process a set number of times to obtain a set number of affine matrices. B2. Calculate the affine matrix using the error formula to obtain the affine matrix with the minimum error. The affine matrix with minimum error The optimal affine matrix for this cluster; the error formula is specifically as follows: B3. Repeat steps B1 and B2 to obtain the optimal affine matrices for several clusters; B4. Compare the optimal affine matrices pairwise to determine whether the ratio of the error values ​​calculated using the matching points within the clusters of the two affine matrices is greater than a preset value. If so, it indicates that the difference between the two optimal affine matrices is too large and they do not satisfy the same affine transformation, so they are not merged. Otherwise, the clusters containing the two optimal affine matrices are merged; The tampered area localization module estimates the affine matrix through the matching points within the clusters, locates the tampered area of ​​the image based on the affine matrix, and outputs the localization result of the tampered area.

Images