Image copy-paste tampering detection method based on bag-of-visual-words model

Abstract

The present invention discloses an image copy-paste tampering detection algorithm based on a visual bag-of-words model, comprising the following steps: normalizing the image size, extracting feature points using a SIFT algorithm; constructing a complex-valued Bessel Fourier moment invariant of a feature point region of interest as a feature of the feature point, matching the feature to obtain a word-level matching result; calculating visual phrases using the geometric relationship between matching feature points, performing maximum pooling to obtain phrase-level features; constructing a heat map based on edge information, using the heat map to weight phrase-level features, directly matching phrase-level features to obtain phrase-level matching results; and executing a post-processing algorithm to obtain a final detection result. The word-level and phrase-level feature description and matching of the present invention produces fewer mismatch pairs for natural images with high self-similarity, which not only reduces the execution time of the algorithm post-processing, but also improves the precision of the algorithm; and the algorithm has better robustness, which improves the recall of the algorithm.

Description

Technical Field

[0001] The invention relates to an image tampering detection method, in particular to an image copy-paste tampering detection method based on a visual bag-of-words model, and belongs to the technical field of digital image authentication. Background Art

[0002] As humans enter the information society, multimedia information is growing at an unimaginable rate. As one of the most important sources of information for humans, images play a key role in the formation of group and individual opinions. The rapid promotion of modern image processing software and personal smart terminals has enabled people to tamper with the content of digital images at a very low cost, which poses a serious threat to the reliability of digital images.

[0003] Copy-paste tampering is a common operation in various types of digital image tampering, in which one or more areas of an image are copied and pasted elsewhere in the same image with the purpose of hiding or emphasizing the object of interest. Experienced tamperers can deceive ordinary observers through these simple operations to achieve illegal or unethical goals. Although the process of copy-paste tampering is very simple, it is not easy to detect it. First, the content of the tampered area comes from the tampered image itself, so its basic physical characteristics (such as lighting and viewing angle) are not significantly different from other areas of the image. In addition, in order to prevent the tampering from leaving visible traces, tamperers usually perform affine transformation operations (including rotation, scaling, mirroring, etc.) on the copied area, and perform conventional signal attacks (including noise, compression, blurring, etc.) on the entire tampered image (or only the copied area), which makes copy-paste tampering detection difficult.

[0004] Copy-paste tampering detection generally relies on reliable feature description operators to represent the local attributes of the image, and matches the above features to find image fragments with the same semantics. Therefore, copy-paste tampering detection usually includes three main steps: feature extraction, feature matching, and post-processing. Feature extraction: calculate the appropriate features of each pixel or a specific part of the key points to represent the field of the pixel; matching: use a certain distance evaluation index to measure the "similarity" of any two features and classify similar features into one category; post-processing: use the matching results obtained in the previous step to accurately locate the actual boundary of the copy-paste tampering. According to whether the local features of the image are extracted from each pixel or only from some selected key points, the existing copy-paste tampering detection technology can be mainly divided into two categories: dense domain (based on dense blocks) and sparse domain (based on feature points). 1) Dense domain (based on dense blocks) method: The region of interest for feature extraction is the dense blocks of the image, which generally have a fixed size, shape, and position. 2) Sparse domain (feature point based) method: The regions of interest for feature extraction are a few selected "salient" regions, the size, shape and position of which generally change with the image content.

[0005] Nowadays, blind forensics technology for image copy-paste tampering has become a research hotspot in academia, but there are still several open problems to be solved in this field, including: 1) For natural images with high self-similarity, the algorithm may produce a large number of false matches, which increases the difficulty of post-processing algorithms; 2) For conventional signal attacks (such as noise), the number of correct matches in the algorithm may be greatly reduced, which reduces the reliability of the detection results. Summary of the invention

[0006] The present invention aims to solve the above-mentioned technical problems existing in the existing copy-paste tampering detection methods and provides an image copy-paste tampering detection method based on a visual word bag model.

[0007] The technical solution of the present invention is an image copy-paste tampering detection method based on a visual bag-of-words model.

[0008] An image copy-paste tampering detection algorithm based on a visual bag-of-words model includes the following steps:

[0009] S1, normalize the image size and use SIFT algorithm to extract feature points;

[0010] S2, constructing the complex-valued Bessel Fourier moment invariant of the feature point interest area as the feature of the feature point, and matching the features to obtain the word-level matching result;

[0011] S3, using the geometric relationship between the matching feature points to calculate the visual phrase, perform maximum pooling to obtain phrase-level features; then, construct a heat map based on edge information, use the heat map to weight the phrase-level features, directly match the phrase-level features, and obtain phrase-level matching results;

[0012] S4. Finally, execute the post-processing algorithm to obtain the final detection result.

[0013] Furthermore, the step S1 includes the following steps:

[0014] S11, initial setting: obtain the image to be detected I, and set the normalized size BZ of the region of interest and the threshold of the feature point set K;

[0015] S12, feature point extraction:

[0016] a. Read in image I, the image size is N I ×M I , resize the image according to the scale factor s,

[0017] This means that the long side of the input image with a smaller resolution is normalized to 3000 pixels, and the input image with a larger resolution is not normalized, so there is no loss of information;

[0018] b. Execute SIFT algorithm on image I to obtain feature point set K = {k k}.

[0019] Furthermore, step S2 includes the following steps:

[0020] S21, feature point k k The region of interest is defined as: Where (x k ,y k ) is the coordinate in the image plane, σ k represents the scale, NoK is the number of feature points; then in the discrete domain of the digital image, the circumscribed square of the region of interest in the digital domain is adjusted to the standard size BZ×BZ to obtain the normalized region of interest Where (i, j) is the coordinate of the image pixel;

[0021] S22. According to the basis function of Bessel Fourier moment: Calculated to be (K+1) 2 The two-dimensional Bessel Fourier moment basis functions {F k [i,j]:(i,j)∈[1,...,BZ] 2};

[0022] S23, normalize all regions of interest Reorganize into a matrix, denoted by KR, where each row corresponds to a normalized region of interest; all two-dimensional Bessel Fourier moment basis functions F k [i,j] is reorganized into a matrix, denoted by BF, where each column corresponds to a basis function; calculate the product of the two matrices: M = KR × BF, where each row m in M k Corresponding to a feature point k k The Bessel Fourier moment of the complex valued invariant in the moment value is combined with the real part, imaginary part and real value to obtain the word-level feature vector

[0023] S24, matching feature vectors, obtaining a word-level matching pair set P w ={(k,k′) k}, where (k, k′) is a pair of matching feature points.

[0024] Furthermore, step S3 includes the following steps:

[0025] S31. Find the nearest neighbors of each word-level matching feature point on the K image plane to form a geometric visual phrase Among them, visual words is called the center word of the geometric visual phrase G, and the remaining K visual words The edge words are called geometric visual phrases G;

[0026] S32, based on the geometric phrase pooling strategy, calculate each feature point k c Geometric phrase features

[0027]

[0028] in yes d dimensions, yes There are d dimensions in , and max() means taking the maximum value;

[0029] S33, for image I = {f(i,j):(i,j)∈[1,...,N I ]×[1,...,M I ]}, calculate its edge image as: I e ={e(i,j):(i,j)∈[1,...,N I ]×[1,...,M I ]}, e(i,j)∈[0,1] is the edge response of image I at (i,j);

[0030] S34, based on edge image I e, calculate the spatial weighted heat map: I w ={w(i,j):(i,j)∈[1,...,N I ]×[1,...,M I ]},

[0031] Among them, w(i,j) is the weighted value of image I at (i,j), which is defined by the accumulation of edge strength of Gaussian blur: G ij is the Gaussian kernel function;

[0032] S35, based on geometric phrase features and spatial weighted heat map I w , calculate the feature point k c The corresponding spatial weighted geometric phrase feature, i.e., phrase-level feature

[0033] S36. Match phrase-level features to obtain a phrase-level feature matching pair set P. p ={(k,k′) k}, where (k, k′) is a pair of matching feature points.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] First, the word-level feature description and matching in step S2 of the present invention and the phrase-level feature description and matching in step S3 produce fewer mismatch pairs for natural images with high self-similarity, which not only reduces the execution time of algorithm post-processing (as shown in Table 2), but also improves the precision of the algorithm (as shown in Table 1);

[0036] Second, the number of correct matching pairs generated by the word-level feature description and matching in step S2 of the present invention and the phrase-level feature description and matching in step S3 for conventional geometric or signal attacks is relatively stable, which indicates that the algorithm is more robust and improves the recall rate of the algorithm (e.g. Figure 3 shown); BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a schematic diagram of geometric phrase pooling according to an embodiment of the present invention;

[0038] Figure 2 is a schematic diagram of edge-based spatial weighting according to an embodiment of the present invention;

[0039] Figure 3 It is a subjective schematic diagram of the detection results of the embodiment of the present invention in the image libraries FAU, GRIP, MICC-F600 and CMH;

[0040] Figure 4 is a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0041] like Figure 4 As shown, the method of the present invention includes four steps: feature point extraction, word-level feature description and matching, phrase feature description and matching, and post-processing.

[0042] Convention: I refers to the image to be detected, (i, j) is the coordinate of the image pixel, N I ×M I is the image resolution, K is the feature point set, is the normalized region of interest of the feature point, ∈ indicates belonging to, BZ indicates the normalized size of the region of interest, F k [i,j] represents the basis function of Bessel Fourier moment, is the word-level feature, P w The word-level matching pair set, G is a geometric visual phrase, I e is the edge image, I w is the spatial weighted heat map, G ij is the Gaussian kernel function, is the phrase-level feature, P p Phrase-level matching pair set, R is the final tampering detection result, max() means taking the maximum value, Represents convolution.

[0043] A method for detecting image copy-paste tampering based on a bag-of-visual-words model comprises the following steps:

[0044] S1, normalize the image size and use SIFT algorithm to extract feature points;

[0045] S11, initial setting: obtain the image to be detected I, and set the threshold BZ = 15, K = 3

[0046] S12, feature point extraction:

[0047] a. Read in image I, the image size is N I ×M I , resize the image according to the scale factor s,

[0048] This means that the long side of the input image with a smaller resolution is normalized to 3000 pixels, and the input image with a larger resolution is not normalized, so there is no loss of information;

[0049] b. Execute SIFT algorithm on image I to obtain feature point set K = {k k}.

[0050] S2, constructing the complex-valued Bessel Fourier moment invariant of the feature point interest area as the feature of the feature point, and matching the features to obtain the word-level matching result;

[0051] S21, feature point k k The region of interest is defined as: Where (x k ,y k ) is the coordinate in the image plane, σ k represents the scale, NoK is the number of feature points; then in the discrete domain of the digital image, the circumscribed square of the region of interest in the digital domain is adjusted to the standard size BZ×BZ to obtain the normalized region of interest Where (i, j) is the coordinate of the image pixel;

[0052] S22. According to the basis function of Bessel Fourier moment: Calculated to be (K+1) 2 The two-dimensional Bessel Fourier moment basis functions {F k [i,j]:(i,j)∈[1,...,BZ] 2};

[0053] S23, normalize all regions of interest Reorganize into a matrix, denoted by KR, where each row corresponds to a normalized region of interest; all two-dimensional Bessel Fourier moment basis functions F k [i,j] is reorganized into a matrix, denoted by BF, where each column corresponds to a basis function; calculate the product of the two matrices: M = KR × BF, where each row m in M k Corresponding to a feature point k k The Bessel Fourier moment of the complex valued invariant in the moment value is combined with the real part, imaginary part and real value to obtain the word-level feature vector

[0054] S24, matching feature vectors, obtaining a word-level matching pair set P w ={(k,k′) k}, where (k, k′) is a pair of matching feature points.

[0055] S3, using the geometric relationship between the matching feature points to calculate the visual phrase, perform maximum pooling to obtain phrase-level features; then, construct a heat map based on edge information, use the heat map to weight the phrase-level features, directly match the phrase-level features, and obtain phrase-level matching results;

[0056] S31. Find the nearest neighbors of each word-level matching feature point on the K image plane to form a geometric visual phrase Among them, visual words is called the center word of the geometric visual phrase G, and the remaining K visual words The edge words are called geometric visual phrases G;

[0057] S32, such as Figure 1 As shown, based on the geometric phrase pooling strategy, each feature point k is calculated c Geometric phrase features

[0058] in yes d dimensions, yes There are d dimensions in , and max() means taking the maximum value;

[0059] S33, such as Figure 2 As shown, for image I = {f(i,j):(i,j)∈[1,...,N I ]×[1,...,M I ]}, calculate its edge image as: I e ={e(i,j):(i,j)∈[1,...,N I ]×[1,...,M I ]}, e(i,j)∈[0,1] is the edge response of image I at (i,j);

[0060] S34, based on edge image I e , calculate the spatial weighted heat map: I w ={w(i,j):(i,j)∈[1,...,N I ]×[1,...,M I ]},

[0061] Among them, w(i,j) is the weighted value of image I at (i,j), which is defined by the accumulation of edge strength of Gaussian blur: G ij is the Gaussian kernel function;

[0062] S35, based on geometric phrase features and spatial weighted heat map I w , calculate the feature point k c The corresponding spatial weighted geometric phrase feature, i.e., phrase-level feature

[0063] S36. Match phrase-level features to obtain a phrase-level feature matching pair set P. p ={(k,k′) k}, where (k, k′) is a pair of matching feature points.

[0064] S4. Finally, the matching result P p ={(k,k′) k}After post-processing, the final tampering detection result R is obtained.

[0065] like Figure 3 It is a subjective schematic diagram of the detection results of the embodiment of the present invention in the image libraries FAU, GRIP, MICC-F600 and CMH; Experimental test and parameter setting: The experiment was performed in the MatlabR2021a environment. The image libraries involved in the experiment are FAU, GRIP, MICC-F600 and CMH, which are all public and can be searched and downloaded online. Table 1 is the pixel-level and image-level objective results of the embodiment of the present invention in the image libraries FAU, GRIP, MICC-F600 and CMH; Table 2 is the CPU time (seconds) and the total number of matches in different feature description / matching stages;

[0066] Table 1

[0067]

[0068]

[0069] Table 2

[0070]

[0071] The specific implementation scheme described above further describes the purpose and technical solutions of the present invention in detail. Those skilled in the art will understand that the above is only a specific implementation scheme of the present invention and is not intended to limit the scope of the present invention. Various modifications can be made in the details without departing from the limitations of the attached claims, which should all be deemed to belong to the scope of protection of the present invention.

Claims

1. A method for detecting image copy-paste tampering based on a bag-of-visual-words model, characterized in that: The steps include: S1, normalize the image size and use SIFT algorithm to extract feature points; S2, constructing the complex-valued Bessel Fourier moment invariant of the feature point interest area as the feature of the feature point, and matching the features to obtain the word-level matching result; S3, using the geometric relationship between the matching feature points to calculate the visual phrase, perform maximum pooling to obtain phrase-level features; then, construct a heat map based on edge information, use the heat map to weight the phrase-level features, directly match the phrase-level features, and obtain phrase-level matching results; S4. Finally, execute the post-processing algorithm to obtain the final detection result; The step S2 comprises the following steps: S21, feature point k k The region of interest is defined as: Where (x k ,y k ) is the coordinate in the image plane, σ k represents the scale, NoK is the number of feature points; then in the discrete domain of the digital image, the circumscribed square of the region of interest in the digital domain is adjusted to the standard size BZ×BZ to obtain the normalized region of interest Where (i, j) is the coordinate of the image pixel; S22. According to the basis function of Bessel Fourier moment: Calculated to be (K+1) 2 The two-dimensional Bessel Fourier moment basis functions {F k [i,j]:(i,j)∈[1,...,BZ] 2 }; S23, normalize all regions of interest Reorganize into a matrix, denoted by KR, where each row corresponds to a normalized region of interest; all two-dimensional Bessel Fourier moment basis functions F k [i,j] is reorganized into a matrix, denoted by BF, where each column corresponds to a basis function; Calculate the product of these two matrices: M = KR × BF, each row m in M k Corresponding to a feature point k k The Bessel Fourier moment of the complex valued invariant in the moment value is combined with the real part, imaginary part and real value to obtain the word-level feature vector S24, matching feature vectors, obtaining a word-level matching pair set P w ={(k,k′) k }, where (k, k′) is a pair of matching feature points; The step S3 comprises the following steps: S31. Find the nearest neighbors of each word-level matching feature point on the K image plane to form a geometric visual phrase Among them, visual words is called the center word of the geometric visual phrase G, and the remaining K visual words The edge words are called geometric visual phrases G; S32, based on the geometric phrase pooling strategy, calculate each feature point k c Geometric phrase features in yes d dimensions, yes There are d dimensions in , and max() means taking the maximum value; S33, for image I = {f(i,j):(i,j)∈[1,...,N I ]×[1,...,M I ]}, calculate its edge image as: I e ={e(i,j):(i,j)∈[1,...,N I ]×[1,...,M I ]}, e(i,j)∈[0,1] is the edge response of image I at (i,j); S34, based on edge image I e , calculate the spatial weighted heat map: I w ={w(i,j):(i,j)∈[1,...,N I ]×[1,...,M I ]}, Among them, w(i,j) is the weighted value of image I at (i,j), which is defined by the accumulation of edge strength of Gaussian blur: G ij is the Gaussian kernel function; S35, based on geometric phrase features and spatial weighted heat map I w , calculate the feature point k c The corresponding spatial weighted geometric phrase feature, i.e., phrase-level feature S36. Match phrase-level features to obtain a phrase-level feature matching pair set P. p ={(k,k′) k }, where (k, k′) is a pair of matching feature points.

2. According to claim 1, the image copy-paste tampering detection method based on the visual word bag model is characterized in that: The step S1 comprises the following steps: S11, initial setting: obtain the image to be detected I, and set the normalized size BZ of the region of interest and the threshold of the feature point set K; S12, feature point extraction: a. Read in image I, the image size is N I ×M I , resize the image according to the scale factor s, This means that the long side of the input image with a smaller resolution is normalized to 3000 pixels, and the input image with a larger resolution is not normalized. No loss of information; b. Execute SIFT algorithm on image I to obtain feature point set K = {k k }.

Images