Cross-view stitching detection method based on coarse-to-fine matching

Abstract

The application provides a cross-view splicing detection method based on coarse-to-fine matching, which comprises the following steps: performing feature extraction on a to-be-detected image to obtain feature points; setting a coarse matching threshold, matching the feature points to obtain a coarse matching set, wherein the coarse matching set comprises correct matching pairs and incorrect matching pairs; modeling the coarse matching set to obtain a mixed model, performing maximum a posteriori estimation on the mixed model, distinguishing the correct matching pairs from the incorrect matching pairs, and obtaining an output correct matching pair set; and positioning a splicing tampering region of the to-be-detected image according to the correct matching pair set to obtain a detection result.

Description

Technical Field

[0001] This invention relates to the field of digital image forensics, and in particular to a cross-view splicing detection method based on coarse-to-fine matching. Background Technology

[0002] With the development of image editing software, people have increasingly used it to edit images before uploading them to social networks and other platforms. While image editing technology has brought many conveniences to people's lives, it also carries potential dangers. If maliciously altered images are used for news reporting, academic fraud, or malicious defamation, they can cause incalculable serious harm, ranging from minor negative impacts on individuals to serious threats to national security. Therefore, digital image forensics technology has emerged.

[0003] Same-source stitching is one of the most common types of digital image manipulation. It involves copying a portion of an image and then pasting it into other areas of the same image to emphasize or conceal specific targets. After same-source stitching, the manipulated image will have at least one highly similar area, creating a strong sense of unnaturalness. Generally, forged images produced through same-source stitching are easily detectable. Therefore, to enhance the realism of the manipulation, forgers stitch the source target together from different perspectives. Compared to traditional same-source stitching, forged images produced through cross-perspective stitching are far more deceptive.

[0004] Image stitching detection schemes for forged images generated by cross-view stitching can be broadly divided into two types based on the stitching type: same-source stitching detection and different-source stitching detection. Same-source stitching tampering detection can be further divided into three types.

[0005] The first type is a detection algorithm based on image patch features. It uses DCT coefficients as image patch features and leverages dictionary sorting for feature matching. PCA, KPCA, and SVD are used to reduce feature dimensionality. Gaussian pyramids are used to decompose the image, and Hu moments are used as image patch features. However, this method has limitations. First, image patch features are generally extracted from overlapping blocks, leading to a large number of features and computational overhead due to the large number of overlapping blocks. Second, image patch features are not robust to post-processing operations such as scaling, making it difficult to effectively detect post-processed images. The second type is a detection algorithm based on keypoint features. Since keypoint extraction relies on points with high entropy values ​​in the image, the distribution of keypoints in the image is relatively sparse. Experiments show that keypoint feature-based detection algorithms have lower computational complexity. The third type is a detection algorithm based on depth features. This end-to-end detection network can distinguish between stitched targets and source targets.

[0006] For heterogeneous image stitching tampering detection, it can be divided into two types based on feature type. The first type is detection algorithms based on traditional statistical features, such as noise patterns, CFA desacrifice features, and operation-related tampering trace features. The second type is detection methods based on deep learning. Early deep learning-based methods divided the image into different blocks and detected tampering by learning the differences between the different blocks. In recent years, researchers at home and abroad have proposed treating image stitching detection as a segmentation task.

[0007] Analysis of same-source stitching tampering detection methods and heterogeneous stitching tampering detection methods reveals that same-source stitching tampering detection methods detect tampering by searching for similar regions in the image. However, when the viewing angle changes, the similarity between the two regions is limited, making robust feature matching difficult. Heterogeneous stitching tampering detection, on the other hand, detects tampering based on the differences in statistical features between the tampered region and the original region. However, in cross-view stitching, the stitching target and the source target have a certain degree of similarity, and their statistical features do not differ significantly, making high-accuracy cross-view stitching detection impossible. Summary of the Invention

[0008] The embodiments of the present invention provide a cross-view splicing detection method based on coarse-to-fine matching to overcome the shortcomings of the prior art.

[0009] To achieve the above objectives, the present invention adopts the following technical solution.

[0010] On the one hand, the present invention provides a cross-view stitching detection method based on coarse-to-fine matching, comprising:

[0011] Feature extraction is performed on the image to be tested to obtain feature points;

[0012] Set a coarse matching threshold, match feature points to obtain a coarse matching set, which includes correct matching pairs and incorrect matching pairs;

[0013] Model the coarse matching set to obtain a mixture model, perform maximum a posteriori probability estimation on the mixture model, distinguish between correct and incorrect matching pairs to obtain the set of correct matching pairs;

[0014] Based on the set of correctly matched pairs, the splicing and tampering areas of the image under test are located, and the detection results are obtained.

[0015] Optionally, locating the stitched and altered areas of the image to be tested includes:

[0016] Perform superpixel segmentation on the image to be tested and filter out image patches containing matching points from the correct matching set;

[0017] Construct the convex hull based on all elements within the image patch;

[0018] Image segmentation is performed on the convex hull to refine the spliced ​​and tampered areas.

[0019] Optionally, the feature descriptors for the feature points include hand-designed features and features extracted by CNNs.

[0020] Optionally, hand-designed features include SIFT and SURF, while CNN-extracted features include ContextDesc and R2D2.

[0021] Optionally, matching methods for multiple feature points include:

[0022] By calculating a feature point s i The Euclidean distances between the feature points and all other feature points form the distance set D = {d1, ..., d2}. n-1};

[0023] Calculate the ratio R of two adjacent distances in the distance set. If R satisfies k Less than the coarse matching threshold, and R k+1 If the value is greater than the coarse matching threshold, then {d1,d2,...,d k The corresponding feature points and feature points s i Matching yields a coarse matching set, which is {Q1, Q2}.

[0024] Optionally, the ratio R of two adjacent distances in the distance set can be calculated based on the following formula:

[0025]

[0026] In the formula, d iLet d be a distance in the distance set. i+1 For distance set with d i The next adjacent distance.

[0027] Optionally, in the fine-matching model, the coordinates of correctly matched pairs follow a Gaussian distribution with mean 0 and variance σ, while the coordinates of incorrectly matched pairs follow a uniform distribution with parameter c. h is the latent variable associated with both correctly and incorrectly matched pairs. n ∈{0, 1}, where 1 represents a correct match and 0 represents an incorrect match. The likelihood function of the mixture model is:

[0028]

[0029] In the formula, p is the posterior probability, and q 1n q 2n Let θ = {f, σ, α} be the coordinates of the matching point, α be the set of unknown parameters, B be the distance, and f be the vector field.

[0030] Optionally, distance B can be calculated based on the following formula:

[0031]

[0032] In the formula, q 1n q 2n Let B be the coordinates of the matching point, B be the distance, and f be the vector field.

[0033] Alternatively, f is obtained by interpolating the coarse matching set and used to fit the correct match.

[0034] Optionally, a smoothing constraint is imposed on f based on the following equation:

[0035]

[0036] In the formula, p(f) is the probability distribution function of f. Let λ be the regenerated kernel Hilbert space, where λ is a positive real number.

[0037] The beneficial effects of this invention are as follows: It improves the robustness of matching by employing a coarse-to-fine matching strategy. Coarse matching retains more matching pairs by appropriately increasing the matching threshold, while fine matching interpolates a vector field onto coarse-matched samples containing incorrect matches, allowing the vector field to fit potentially correct matching samples. This invention retains more cross-view region matching pairs by interpolating a smooth vector field onto coarse-matched samples containing incorrect matches, and achieves pixel-level localization of tampered regions through graph cut, effectively improving the performance of cross-view stitching detection.

[0038] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating a cross-view stitching detection method based on coarse-to-fine matching provided in an embodiment of the present invention.

[0041] Figure 2 A schematic diagram illustrating how iterative changes are judged as incorrect matching categories in an embodiment of the present invention;

[0042] Figure 3 The robustness test line graph provided in the embodiment of the present invention. Detailed Implementation

[0043] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0044] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0045] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0046] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0047] Example

[0048] like Figure 1 As shown, this embodiment provides a cross-view stitching detection method based on coarse-to-fine matching, including the following steps:

[0049] Step 1: Extract features from the image to be tested to obtain feature points.

[0050] Specifically, feature extraction involves extracting a large number of feature points from the image to be tested for subsequent matching. Feature descriptors can include two types: one is features designed manually, such as SIFT and SURF; the other is features extracted by CNNs, such as ContextDesc and R2D2.

[0051] Step 2: Set a coarse matching threshold, match feature points to obtain a coarse matching set, which includes correct matching pairs and incorrect matching pairs.

[0052] Specifically, coarse matching sets a suitable threshold T for the extracted feature points. mat Perform matching for each feature point s i The Euclidean distances between the feature points and all other feature points form the distance set D = {d1, ..., d2}. n-1 Let the ratio of the current distance to the next distance be:

[0053]

[0054] If R is satisfied k <T mat And R k+1 >T mat Then {d1, d2, ..., d} k The corresponding feature points and s i After coarse matching, the matching set {Q1, Q2} is obtained, where Q is the match set. i It also represents the initial category of the feature points.

[0055] Step 3: Model the coarse matching set to obtain a mixture model, perform maximum a posteriori probability estimation on the mixture model, distinguish between correct and incorrect matching pairs, and obtain the set of correct matching pairs.

[0056] Specifically, fine matching involves interpolating a vector field f: Q1→Q2 from the coarse matching set, which includes incorrect matches, to fit the potential correct matches as closely as possible. After modeling, it is assumed that the coordinates of correct matches follow a Gaussian distribution with mean 0 and variance σ, while incorrect matches follow a uniform distribution with parameter c. A latent variable h is associated with all matching pairs. n ∈{0, 1}, where 1 represents a correct match and 0 represents an incorrect match. The likelihood function of the model is:

[0057]

[0058] Where θ = {f, σ, α} is the set of unknown parameters, α is the mixing coefficient, and the distance metric is cosine distance.

[0059]

[0060] Additionally, a smoothing constraint is applied to f:

[0061]

[0062] Where p(f) is the probability distribution function of f. Let λ represent the reproducing kernel Hilbert space, where λ is a positive real number. The EM algorithm can be used to find the maximum a posteriori solution, thereby estimating the parameters and distinguishing between correct and incorrect matches.

[0063] Although each feature point is assigned an initial category after coarse matching, it cannot be guaranteed that all feature points whose coordinates fall within the stitching region will have the same initial category. This can lead to some matching pairs being misclassified as incorrect matches. This embodiment employs an iterative method to change the categories judged as incorrect matches. For example... Figure 2 As shown, after each match pair indicated as an incorrect match is reclassified, the parameters are re-estimated until the class of the matching points tends to stabilize, resulting in a set of matching pairs {V1, V2}. To further delete incorrect matches, a clustering method based on spatial coordinates is used to classify isolated matches as incorrect matches.

[0064] Specifically, including:

[0065] To create an empty set U1 and remove any element from V1 and place it into U1, the condition for merging the remaining elements from V1 into U1 is:

[0066]

[0067] In the formula, K is the number of elements in U1, ω is the scaling factor, and if the Euclidean distance between the element to be merged and all elements in U1 is less than γ, then ξ i =1, otherwise 0. Next, remove the elements merged into U1 from V1 and create a new set, repeating the previous step until V1 becomes an empty set. Finally, classify the matching pairs corresponding to sets with fewer than 3 elements as isolated matches and delete them, obtaining the correctly matched set.

[0068] Step 4: Based on the set of correct matching pairs, locate the spliced ​​and tampered areas in the image to be tested and obtain the detection results, including the following sub-steps:

[0069] Step 4.1: Perform superpixel segmentation on the image to be tested and select image patches containing matching points from the correct matching set. Specifically, perform superpixel segmentation on the image to be tested and select image patches Pa containing matching points. i .

[0070] Step 4.2: Construct the convex hull based on all elements within the image patch. Specifically, using Pa... i The convex hull is constructed based on all elements within it.

[0071] Step 4.3: Perform image segmentation on the convex hull to refine the tampered area. Specifically, perform image cutting on the convex hull to further refine the tampered area and improve the pixel-level positioning accuracy of the tampered area.

[0072] In this embodiment, two datasets were used for the test images. The first dataset was the manually created cross-view stitching dataset CVCMD, which collected 6600 images containing various attacks, with positive and negative examples each accounting for half. The second dataset was CoMoFoD, which contained 200 positive samples and 200 negative samples, with a resolution of 512×512.

[0073] For descriptors with different features, five features were selected for comparison in this embodiment, and the results are as follows:

[0074] As shown in Table 1.

[0075]

[0076] Table 1. Detection performance using different features on the CVCMD dataset.

[0077] As shown in Table 1, the selected feature descriptors are SIFT, SURF, BRISK, R2D2, and ContextDesc. By comparing the performance of two matching methods—Coarse+RANSAC and the coarse-to-fine matching strategy proposed in this embodiment—it can be seen that for all feature descriptors, coarse-to-fine matching outperforms Coarse+RANSAC in both inner ratio and F1-image. SIFT features achieve the best performance on F1-image, while ContextDesc achieves the best performance on inner ratio. Since SIFT features offer the best overall performance, they are chosen for the feature extraction part.

[0078] This embodiment uses ablation experiments to compare five methods: "Coarse" represents coarse matching; "Coarse+RANSAC" represents RANSAC to remove mismatches from coarse matching samples; "Coarse+Field" represents smooth vector field interpolation to remove mismatches from coarse matching samples; "Coarse+Field+Isolate" represents smooth vector field interpolation to remove isolated matching pairs from coarse matching samples; and "Coarse+Field+Category" represents category adjustment based on "Coarse+Field". The comparison results are shown in Table 2.

[0079]

[0080] Table 2 Ablation experiments on the CVCMD dataset

[0081] As can be seen from Table 2, the coarse-to-fine matching strategy achieved the best overall performance of 95.80% Inner ratio, 100.00% Precision and 96.77% F1, which further illustrates the excellent effect of the cross-view stitching detection method based on coarse-to-fine matching provided in this embodiment.

[0082] This embodiment compares its method with the homology concatenation detection method on the CVCMD dataset. The comparison results are as follows:

[0083] As shown in Table 3.

[0084]

[0085] Table 3 Comparison with homology splicing detection methods on the CVCMD dataset.

[0086] As can be seen from Table 3, the cross-view stitching detection method based on coarse-to-fine matching provided in this embodiment achieves the best performance of 96.77% F1-image and 77.39% F1-pixel.

[0087] In addition, to further verify the robustness of the cross-view stitching detection method based on coarse-to-fine matching provided in this embodiment, four sets of robustness experiments were conducted in this embodiment. The robustness experiment line graph is shown below. Figure 3 As shown, the four robustness experiments cover JPEG compression, noise addition, rotation, and scaling. For JPEG compression, the cross-view splicing detection method based on coarse-to-fine matching provided in this embodiment exhibits good robustness for compression quality factors of 50-100, with a slight performance decrease when the quality factor exceeds 50. For noise addition, the performance of most comparison methods is relatively stable. For rotation and scaling, the performance of segmentation fluctuates significantly. In summary, the cross-view splicing detection method based on coarse-to-fine matching provided in this embodiment demonstrates good robustness against these four attacks.

[0088] In addition, this embodiment uses the CoMoFoD dataset to compare with the same source concatenation algorithm, and the comparison results are shown in Table 4.

[0089]

[0090] Table 4 Comparison with homology stitching detection methods on the CoMoFoD dataset.

[0091] As shown in Table 4, Hierarchical and the cross-view splicing detection method based on coarse-to-fine matching provided in this embodiment achieved the highest detection accuracy in F1-pixel and F1-image metrics, respectively, demonstrating the excellent detection performance of the cross-view splicing detection method based on coarse-to-fine matching provided in this embodiment against traditional copy-paste tampering.

[0092] Finally, this embodiment is compared with the heterogeneous splicing algorithm, and the comparison results are shown in Table 5.

[0093]

[0094] Table 5 Comparison with heterogeneous splicing detection methods on the CVCMD dataset.

[0095] Considering that heterogeneous stitching algorithms theoretically cannot locate the source target region, only the stitching region is statistically analyzed when calculating the pixel-level positioning accuracy. As shown in Table 5, the cross-view stitching detection method based on coarse-to-fine matching provided in this embodiment has the lowest false alarm rate and the highest accuracy in locating the tampered region, proving the effectiveness of the coarse-to-fine matching strategy.

[0096] In summary, this invention improves the robustness of matching through a coarse-to-fine matching strategy. Coarse matching retains more matching pairs by appropriately increasing the matching threshold, while fine matching interpolates the vector field of coarse-matched samples containing erroneous matches, allowing the vector field to fit potentially correct matching samples. This invention retains more cross-view region matching pairs by interpolating a smooth vector field from coarse-matched samples containing erroneous matches, and achieves pixel-level localization of tampered regions through graph cut, effectively improving the performance of cross-view stitching detection.

[0097] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0098] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for method or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the description of the method embodiments. The method and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0099] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A cross-view stitching detection method based on coarse-to-fine matching, characterized in that, include: Feature extraction is performed on the image to be tested to obtain feature points; Set a coarse matching threshold, match the feature points to obtain a coarse matching set, which includes correct matching pairs and incorrect matching pairs; The coarse matching set is modeled to obtain a hybrid model. The hybrid model is then subjected to maximum a posteriori probability estimation to distinguish between the correct matching pairs and the incorrect matching pairs, thereby obtaining a set of correct matching pairs. Based on the set of correct matching pairs, the splicing and tampering regions of the image under test are located, and the detection results are obtained. In the hybrid model, the correctly matched coordinates follow a mean of 0 and a variance of 0. The Gaussian distribution, wherein the mismatch follows a parameter of . The uniform distribution of the correct and incorrect matches, and the associated latent variables. 1 represents a correct match, and 0 represents a wrong match. The likelihood function of the mixture model is: In the formula, p is the posterior probability, and q 1n q 2n To match the coordinates of the points, For a set of unknown parameters, B is the mixing coefficient, and B is the distance. It is a vector field; The distance B is obtained based on the following formula: In the formula, q 1n q 2n To match the coordinates of the points, The correct match is obtained by interpolating the coarse matching set.

2. The method according to claim 1, characterized in that, Locating the spliced ​​and tampered region of the image under test includes: The image to be tested is segmented by superpixels, and image blocks containing matching points from the correct matching set are selected. Construct a convex hull based on all elements within the image block; The convex hull is segmented to refine the spliced ​​and tampered area.

3. The method according to claim 2, characterized in that, The feature descriptors for the feature points include hand-designed features and features extracted by CNN.

4. The method according to claim 3, characterized in that, The hand-designed features include SIFT and SURF, and the CNN-extracted features include ContextDesc and R2D2.

5. The method according to claim 2, characterized in that, The matching method for the multiple feature points includes: By calculating one of the feature points The Euclidean distances between the feature points and all other feature points yield a distance set. ; Calculate the ratio R of two adjacent distances in the distance set. If R satisfies k Less than the coarse matching threshold, and R k+1 If it is greater than the coarse matching threshold, then The corresponding feature points and the feature points Matching is performed to obtain the coarse matching set, which is: .

6. The method according to claim 5, characterized in that, The ratio R of two adjacent distances in the distance set is calculated based on the following formula: In the formula, d i Let d be a distance in the set of distances. i+1 For the distance set with d i The next adjacent distance.

7. The method according to claim 1, characterized in that, Based on the following formula Apply smoothing constraints: In the formula, for The probability distribution function, For the regenerated kernel Hilbert space, It is a positive real number.

Images