A constrained image stitching detection method, device and medium based on key depth feature matching

Abstract

The present invention discloses a constrained image splicing detection method, device and medium based on key depth feature matching. The method comprises: outputting feature tensors, reliability tensors and repeatability tensors of suspicious splicing tampered images and suspicious tampered source images respectively through a full convolution key depth feature extraction network; screening key features in the feature tensor through the reliability tensor and the repeatability tensor to obtain feature descriptors and key point positions of key depth features; fusing the feature descriptors and key point positions of key depth features based on a multi-layer perceptron to output comprehensive features corresponding to each image; calculating feature maps of a matching neural network layer by layer through attention weighting based on the comprehensive features; generating a score matrix based on the output feature map of the matching neural network; expanding rows and columns on the basis of the score matrix, iteratively normalizing the rows and columns of the expanded score matrix, and screening features with high probability in the distribution matrix to obtain the optimal matching result.

Description

Technical Field

[0001] The present invention relates to the technical field of image tampering evidence collection, and more specifically to a constrained image splicing detection method, device and medium based on key depth feature matching. Background Art

[0002] The main goal of constrained image splicing detection is to match the tampered image to be analyzed with the suspected tampered source image, determine whether there is a correlation between the two, and determine the tampered area and tampered source area in the image pair. DMVN (Deep Matching and Validation Network) is the first method based on deep learning specifically for constrained image splicing detection. This method can give the correlation between two input images and locate the tampered and tampered source areas, but this method has poor detection capabilities for small tampered areas. The DMAC network improves this by using a variety of technical means such as dilated convolution, skip design, and dilated spatial pyramid pooling to improve the detection capability of small tampered areas. AttentionDM introduces encoder-decoder structure, attention module, and normalization module based on DMAC to further improve the effect. MSTAF (Multi-scale Target-Aware Framework) effectively improves the detection and positioning capabilities of tampered areas of different scales by combining multi-scale target-aware attention mechanism with feature extraction and correlation matching.

[0003] However, DMVN, DMAC, AttentionDM, and MSTAF all have insufficient processing capabilities for high-resolution images. When the tampered area in the image is small or the tampered area has undergone complex transformations, the detection capabilities are weak: (1) The deep features used still have obvious spatial information loss; (2) Simple dot product operations are used in the feature map association calculation module. These calculation methods cannot effectively utilize the deep feature information, further causing the loss of feature information.

[0004] Therefore, how to provide a constrained image stitching detection method based on key depth feature matching is a problem that those skilled in the art urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a constrained image stitching detection method, device and medium based on key depth feature matching, which alleviates the problem of spatial information loss of depth features, realizes effective processing of high-resolution images, fine-grained detection of small tampered areas and robust detection of strongly transformed areas.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A constrained image splicing detection method based on key depth feature matching, comprising:

[0008] The image to be detected and the suspected tampered source image are taken as input respectively, and the feature tensor, reliability tensor and repeatability tensor of each image are outputted through the fully convolutional key deep feature extraction network;

[0009] The key features are screened in the feature tensor through the reliability tensor and the repeatability tensor to obtain the feature descriptors and key point positions of the key depth features;

[0010] Based on the multi-layer perceptron, the feature descriptors and key point positions of the key deep features are fused to output the comprehensive features corresponding to each image;

[0011] Based on the comprehensive features, the feature map of the matching neural network is calculated layer by layer through attention weighting, and the final enhanced feature map is output;

[0012] Generate a score matrix based on the output feature map of the matching neural network;

[0013] Expand the rows and columns based on the score matrix to explicitly assign unmatched key points, and output the expanded score matrix;

[0014] The allocation matrix is ​​obtained by iteratively normalizing the rows and columns of the extended score matrix;

[0015] By screening the features with high probability in the allocation matrix, the optimal matching result is obtained.

[0016] Preferably, the image to be detected and the suspected tampering source image are respectively used as input, and the feature tensor, reliability tensor and repeatability tensor of each image are correspondingly output through the fully convolutional key deep feature extraction network, specifically including:

[0017] Select the image to be detected I and the suspected tampering source image I' of size W×H, where W represents the height of the input image and H represents the width of the input image. After two 3×3 convolutions, each convolution is followed by batch normalization and ReLU activation function, and the primary feature f1 is output. The primary feature f1 is then fused through a 1×1 convolution to obtain the feature map f2.

[0018] The primary feature f1 is processed through a 3×3 dilated convolution layer and a 3×3 convolution layer, and batch normalization and ReLU activation are performed after each convolution to obtain the feature map f3; the feature maps f2 and f3 are sent to a SE module to perform channel weighting and feature mapping on the feature channels to obtain the feature map f5;

[0019] The feature map f5 is separated into two feature maps with 64 channels, and the two maps are added item by item and then output through a 1×1 convolution layer to obtain the feature map f6. The feature map f3 is obtained by a 3×3 hole convolution layer, a 3×3 convolution layer, batch normalization and ReLU activation to obtain the feature map f4. The feature maps f4 and f6 are added item by item to achieve feature fusion, and the output B is generated after passing through three consecutive 2×2 convolution layers.

[0020] Further processing of output B yields three tensors of the network output: Normalization obtains a feature tensor X of size W×H×C, where C represents the number of channels; 2 The element is squared and 1×1 convolved, and the output is converted into a probability distribution through the Softmax function to obtain a repeatability tensor S of size W×H×1 and a reliability tensor R of size W×H×1, where Normalization means normalizing the Euclidean norm of each eigenvector, x 2 Element-wise squaring squares every element in a matrix.

[0021] Preferably, the key features are screened in the feature tensor by using the reliability tensor and the repeatability tensor to obtain the feature descriptors and key point positions of the key depth features, specifically including:

[0022] Through repeatability S and reliability R, sparse, repeatable, and highly discriminative key deep features are selected from the feature tensor X. Determined by finding the local maximum of the repeatability S and checking whether it exceeds the corresponding reliability level R;

[0023] Through the above steps, the input image to be detected I and the suspected tampering source image I' are used to obtain the feature descriptor of the image to be detected containing M key depth features. and its coordinate position P, the feature descriptor of the suspected tampered source image contains N key depth features and its coordinate position P'.

[0024] Preferably, the comprehensive feature calculation formula corresponding to the image to be detected is:

[0025]

[0026] in, (1) F i Represents the comprehensive features of a key point i in the image to be detected, The feature descriptor representing the key depth feature corresponding to the image to be detected I, P i Indicates the position corresponding to the key point i, MLP enrepresents a multi-layer perceptron encoder, i = 1, 2, ... M, M represents the number of key feature points of the image to be detected;

[0027] The calculation formula for the comprehensive features corresponding to the suspected tampered source image is:

[0028]

[0029] in, (1) F' j Represents the comprehensive features of a key point j in the suspected tampered source image, The feature descriptor representing the key depth features corresponding to the suspected tampered source image I', P' j Represents the position corresponding to the key point j, j = 1, 2, ... N, N represents the number of key feature points of the suspected tampered source image.

[0030] Preferably, the feature map of the matching neural network is calculated layer by layer through attention weighting based on the comprehensive features, and the final enhanced feature map is output, which specifically includes:

[0031] The initial input feature matrix of the image to be detected is (1) F, the initial input feature matrix of the suspected tampered source image is (1) F' query, query (l) Q, key (l) K, value (l) V is the matrix required for the attention calculation of the lth layer of the image to be detected, (l) Q', (l) K', (l) V' is the matrix required for the attention calculation of the lth layer of the suspected tampered source image, (l) F and (l) F' represents the intermediate representation matrix of the lth iteration, l ≥ 1, and the calculation method is selected according to the number of iteration layers l;

[0032] If l is an odd number, (l) F and (l) F' performs self-attention calculations respectively and calculates (l) Q, (l) K, (l) V and (l) Q', (l) K', (l) V' matrix, according to (l) Q and (l) K gets the self-attention weight (l) A self ,according to (l) Q' and (l) K'get (l) A' self , the attention value is calculated by the attention weight(l) O and (l) O', the output updated features are (l+1) F= (l) F+ (l) MLP( (l) F|| (l) O) and (l+1) F'= (l) F'+ (l) MLP( (l) F'|| (l) O'); (l) MLP represents the multi-layer perceptron of the lth layer, (·||·) represents splicing along the channel direction;

[0033] If l is even, (l) F and (l) F' performs two-way mutual attention calculation and calculates (l) Q, (l) K, (l) V and (l) Q', (l) K', (l) V' matrix, according to (l) Q and (l) K gets the mutual attention weight (l) A cross ,according to (l) Q' and (l) K' gets the mutual attention weight (l) A' cross , calculated by mutual attention weights (l) O and (l) O', the output updated features are (l+1) F= (l) F+ (l) MLP( (l) F|| (l) O) and F' (l+1) = (l) F'+ (l) MLP( (l) F'|| (l) O');

[0034] l starts from 1 until the iteration reaches the required number of network layers L, and finally outputs the feature matrix of the image to be detected (L) F and the feature matrix of the suspected tampered source image (L) F'.

[0035] Preferably, the score matrix calculation formula is:

[0036] S=< (L) F, (L) F'>

[0037] Where S represents the score matrix and <·,·> represents the inner product operation.

[0038] Preferably, assuming that the image to be detected and the suspected tampered source image have M and N key points respectively, let the score matrix S∈R M×N Represents the matching scores between all possible key point pairs, expanding the score matrix for

[0039] Preferably, the allocation matrix is ​​obtained by iteratively normalizing the rows and columns of the extended score matrix, which specifically includes:

[0040] Convert the expanded score matrix to a non-negative matrix using an exponential function ∈ is the regularization parameter,

[0041] Introducing vectors (1) u and vector (1) v, initialization (1) u is a full 1 column vector of length M+1, initialized (1) v is a column vector of length N+1 containing all 1s. (t) u represents the vector u of the tth iteration, (t) v represents the vector v of the tth iteration, (t+1) u and (t+1) v represents the vector u and vector v obtained in the next round of iteration, t≥1; the target row margin vector r is a full-one column vector of length M+1, and the target column margin vector c is a full-one column vector of length N+1;

[0042] At the beginning of the iteration, calculate Where × represents matrix multiplication, Indicates that the vector is used in the tth iteration (t) Pair Matrix The result after scaling is Perform row normalization;

[0043] calculate Where T represents transpose, Indicates that the vector is used in the tth iteration (t) u pair matrix The result after scaling is Perform column normalization;

[0044] The above two steps of normalization are performed alternately until the vectors u and v converge or the preset number of iterations is reached, and the matrix is ​​obtained. Where diag(·) means generating a diagonal matrix from a vector, that is, the main diagonal elements of the generated matrix are the original vector values, and the rest are 0;

[0045] From the matrix Remove the extended rows and columns to obtain the allocation matrix

[0046] A computer device comprises: a memory and a processor, wherein a computer program that can be run on the processor is stored in the memory, and when the processor executes the computer program, the steps of a constrained image splicing detection method based on key depth feature matching are implemented.

[0047] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a constrained image splicing detection method based on key depth feature matching.

[0048] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a constrained image stitching detection method, device and medium based on key depth feature matching, which has the following advantages:

[0049] (1) Using key depth features for constrained image stitching detection. The present invention proposes to study key depth features, construct a non-downsampling, weighted residual enhanced, lightweight fully convolutional key depth feature extraction network, screen and provide more discriminative, sparse depth features, and effectively improve the ability to process high-resolution images and detect small tampered areas.

[0050] (2) A joint optimization model for sparse key depth feature extraction and matching is proposed. The present invention uses a deep learning network to perform deep correlation mining on sparse key depth features, thereby achieving refined matching and enhancing the adaptability of the key depth feature extraction network and matching model in constrained image stitching detection scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0052] Figure 1 A flow chart of a constrained image stitching detection method based on key depth feature matching provided by the present invention.

[0053] Figure 2A schematic diagram of a constrained image stitching detection method based on key depth feature matching provided by the present invention.

[0054] Figure 3 This is the structure diagram of the full convolution key depth feature extraction network provided by the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The embodiment of the present invention discloses a constrained image splicing detection method based on key depth feature matching, such as Figure 1 and Figure 2 As shown, including:

[0057] (1) The image to be detected and the suspected tampered source image are taken as input respectively, and the feature tensor, reliability tensor and repeatability tensor of each image are outputted through the fully convolutional key deep feature extraction network;

[0058] Specifically, select the image to be detected I and the suspected tampering source image I' of size W×H and input them into Figure 3 In the key deep feature extraction network shown in the figure, the image passes through the first set of convolutional layers (including two 3×3 convolutions), each convolution is followed by batch normalization and ReLU activation function, and the primary feature f1 is output. The primary feature f1 is then fused through a 1×1 convolution to obtain the feature map f2.

[0059] The primary feature f1 is processed through a 3×3 dilated convolution layer and a 3×3 convolution layer, and batch normalization and ReLU activation are performed after each convolution to obtain the feature map f3; the feature maps f2 and f3 are sent to a SE module to perform channel weighting and feature mapping on the feature channels to obtain the feature map f5;

[0060] The feature map f5 is separated into two feature maps with 64 channels, and the two maps are added item by item and then output through a 1×1 convolution layer to obtain the feature map f6. The feature map f3 is obtained by a 3×3 hole convolution layer, a 3×3 convolution layer, batch normalization and ReLU activation to obtain the feature map f4. The feature maps f4 and f6 are added item by item to achieve feature fusion, and the output B is generated after passing through three consecutive 2×2 convolution layers.

[0061] Further processing of output B yields three tensors of the network output: Normalization obtains a feature tensor X of size W×H×C, where C represents the number of channels; 2 The element is squared and 1×1 convolved, and the output is standardized by the Softmax function to obtain a repeatability tensor S of size W×H×1 and a reliability tensor R of size W×H×1, where Normalization means normalizing the Euclidean norm of each eigenvector, x 2 Element-wise squaring squares every element in a matrix.

[0062] (2) Filter key features in the feature tensor X through the reliability tensor R and the repeatability tensor S to obtain the feature descriptors and key point positions of the key depth features;

[0063] The specific steps for screening key deep features are: through repeatability S and reliability R, sparse, repeatable, and highly discriminative key deep features can be screened out in the feature tensor X. It can be determined by finding the local maximum of repeatability S and checking whether it exceeds the threshold of the corresponding reliability R. Specifically, for the two input images I and I', the feature descriptors of the images to be detected containing M key depth features can be obtained respectively. and the key point position P, and the N key depths contained in the suspected tampered source image

[0064] Feature Descriptor And the key point position P'.

[0065] (3) Based on the multi-layer perceptron, the feature descriptors and key point positions of the key deep features are fused to output the comprehensive features corresponding to each image;

[0066] Specifically, the comprehensive feature calculation formula corresponding to the image to be detected is:

[0067]

[0068] in, (1) F i Represents the comprehensive features of a key point i in the image to be detected, The feature descriptor representing the key depth feature corresponding to the image to be detected, P i Indicates the position corresponding to the key point i, i = 1, 2, ... M, M represents the number of key feature points of the image to be detected, MLP en represents a multi-layer perceptron encoder;

[0069] The calculation formula for the comprehensive features corresponding to the suspected tampered source image is:

[0070]

[0071] in, (l) F' j Represents the comprehensive features of a key point j in the suspected tampered source image, The feature descriptor representing the key deep features corresponding to the suspected tampered source image, P' j Represents the position corresponding to the key point j, j = 1, 2, ... N, N represents the number of key feature points of the suspected tampered source image.

[0072] (4) The feature map of the matching neural network is calculated layer by layer through the key point information, and the feature value is iteratively updated using the connectivity calculation. In each attention operation, the correlation is aggregated according to the attributes of the key points, considering both the correlation between the internal features of the image and the correspondence between the features between images;

[0073] Specifically, the feature maps of the matching neural network are calculated layer by layer. The following is a detailed explanation of the operations used in the iterative process.

[0074] The initial input feature matrix of the image to be detected is (1) F, the initial input feature matrix of the suspected tampered source image is (1) F' query, query (l) Q, key (l) K, value (l) V is the matrix required for the attention calculation of the lth layer of the image to be detected, (l) Q', (l) K', (l) V' is the matrix required for the attention calculation of the lth layer of the suspected tampered source image, (l) F and (l) F' represents the intermediate representation matrix of the lth iteration, l ≥ 1, the calculation method is selected according to the number of iteration layers l, l is initialized to 1; let (l) W Q , (l) W K and (l) W V as well as (l) b1. (l) b2 and (l) b3 is the initial parameter matrix and bias term for the attention calculation of the lth layer.

[0075] If l is an odd number, (l) F and (l) F' performs self-attention calculations separately:

[0076] right (l) F performs self-attention calculation: calculate respectively (l) Q, (l) K, (l) V Matrix: (l) Q=(l) F (l) W Q + (l) b1, (l) K= (l) F (l) W K + (l) b2, (l) V= (l) F (l) W V + (l) b3. Then calculate the self-attention weight matrix (l) A self , where d k yes (l) The feature dimension of K, then calculate the attention value (l) O: (l) O= (l) A self (l) V.

[0077] right (l) F′ performs self-attention calculation: calculate respectively (l) Q′, (l) K′, (l) V′ matrix: (l) Q′= (l) F′ (l) W Q + (l) b1, (l) K′= (l) F′ (l) W K + (l) b2, (l) V′= (l) F′ (l) W V + (l) b3. Then calculate the self-attention weight matrix (l) A′ self , where d k′ yes (l) The feature dimension of K′, then calculate the attention value (l) O′: (l) O′= (l) A′ self (l) V′.

[0078] Update the feature matrix for the next layer input: (l+1) F= (l) F+ (l) MLP( (l) F|| (l)O), (l+1) F′= (l) F′+ (l) MLP( (l) F′|| (l) O′). (·||·) indicates splicing along the channel direction, (l) MLP represents the multi-layer perceptron used in layer l.

[0079] If l is even, (l) F and (l) F′ performs bidirectional mutual attention calculation:

[0080] right (l) F and (l) F′ performs mutual attention: calculate (l) Q, (l) K′, (l) V′ matrix: (l) Q= (l) F (l) W Q + (l) b1, (l) K′= (l) F′ (l) W K + (l) b2, (l) V′= (l) F′ (l) W V + (l) b3. Then calculate the mutual attention weight matrix (l) A cross And the attention value (l) O: (l) O= (l) A cross (l) V′.

[0081] right (l) F′ and (l) F performs mutual attention: calculate (l) Q′, (l) K, (l) V Matrix: (l) Q′= (l) F′ (l) W Q + (l) b1, (l) K= (l) F (l) W K + (l) b2, (l) V= (l) F (l)W V + (l) b3. Then calculate the mutual attention weight matrix (l) A′ cross And the attention value (l) O′: (l) O′= (l) A′ cross (l) V.

[0082] Update the feature matrix for the next layer input: (l+1) F= (l) F+ (l) MLP(F (l) || (l) O), (l+1) F′= (l) F′+ (l) MLP( (l) F′|| (l) O′). (·||·) indicates splicing along the direction of the number of channels.

[0083] After L layers of iteration, the final output feature representation (L) F and (L) F', where the image to be detected is output as (L) F, the output of the suspected tampered source image is (L) F'.

[0084] (5) Generate a score matrix by matching the correlation information of feature descriptors;

[0085] Specifically, using the obtained (L) F and (L) F' computes a score for each possible pair of keypoint matches between the two images. This is done by calculating (L) F and (L) F' is completed by the inner product, forming an M×N score matrix S, S=< (L) F, (L) F'>, where <·,·> represents the inner product operation, and M and N represent the number of key points of the image to be detected and the suspected tampered source image, respectively.

[0086] (6) Expand the rows and columns based on the score matrix to explicitly assign unmatched key points, and output the expanded score matrix, which means that each key point can be matched to a key point in another image. Such an expanded score matrix is ​​recorded as

[0087] (7) Obtaining a distribution matrix by iteratively normalizing the rows and columns of the extended score matrix;

[0088] Specifically, the expanded score matrix is ​​converted into a non-negative matrix through an exponential function ∈ is the regularization parameter, represents the extended score matrix;

[0089] Introducing vectors (1) u and vector (1) v, initialization (1) u is a column vector of length M+1 containing all 1s. (1) v is a column vector of length N+1 containing all 1s. (t) u represents the vector u of the tth iteration, (t) v represents the vector v of the tth iteration, (t+1) u and (t+1) v represents the vector u and vector v obtained in the next iteration, respectively (t≥1). r is the target row marginal vector, c is the target column marginal vector, these marginal vectors ensure that the rows and columns of the allocation matrix are normalized, that is, the sum of each row and each column is equal to these predefined vectors, since the ideal allocation matrix is ​​1 in both rows and columns, the target row marginal vector r is a full 1 column vector of length M+1, and the target column marginal vector c is a full 1 column vector of length N+1;

[0090] At the beginning of the iteration, calculate Where × represents matrix multiplication, Indicates that the vector is used in the tth iteration (t) v pair matrix The result after scaling is Perform row normalization; calculate Where T represents transpose, Indicates that the vector is used in the tth iteration (t) u pair matrix The result after scaling is Perform column normalization;

[0091] The above two steps of normalization are performed alternately until the vectors u and v converge or the preset number of iterations is reached, and the matrix is ​​obtained. Where diag(·) means generating a diagonal matrix from a vector, that is, the main diagonal elements of the generated matrix are the original vector values, and the rest are 0. Remove the extended rows and columns to obtain the allocation matrix

[0092] (8) The allocation matrix provides an allocation scheme, in which each element represents the probability of a pair of key points matching. By screening the features with high probability in the allocation matrix, the optimal matching result is obtained.

[0093] This embodiment provides a computer device, including: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of a constrained image stitching detection method based on key depth feature matching are implemented.

[0094] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a constrained image stitching detection method based on key depth feature matching are implemented.

[0095] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc. Various media that can store program codes.

[0096] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0097] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A constrained image stitching detection method based on key deep feature matching, characterized in that: include: The image to be detected and the suspected tampered source image are taken as input respectively, and the feature tensor, reliability tensor and repeatability tensor of each image are outputted through the fully convolutional key deep feature extraction network; The key features are screened in the feature tensor through the reliability tensor and the repeatability tensor to obtain the feature descriptors and key point positions of the key depth features; Based on the multi-layer perceptron, the feature descriptors and key point positions of the key deep features are fused to output the comprehensive features corresponding to each image; Based on the comprehensive features, the feature map of the matching neural network is calculated layer by layer through attention weighting, and the final enhanced feature map is output; Generate a score matrix based on the output feature map of the matching neural network; Expand the rows and columns based on the score matrix to explicitly assign unmatched key points, and output the expanded score matrix; The allocation matrix is ​​obtained by iteratively normalizing the rows and columns of the extended score matrix; By screening the features with high probability in the allocation matrix, the optimal matching result is obtained; The image to be detected and the suspected tampered source image are taken as input respectively, and the feature tensor, reliability tensor and repeatability tensor of each image are outputted through the fully convolutional key deep feature extraction network, including: Select the image to be detected I and the suspected tampering source image I' of size W×H, where W represents the height of the input image and H represents the width of the input image. After two 3×3 convolutions, each convolution is followed by batch normalization and ReLU activation function, and the primary feature f1 is output. The primary feature f1 is then fused through a 1×1 convolution to obtain the feature map f2. The primary feature f1 is processed through a 3×3 dilated convolution layer and a 3×3 convolution layer, and batch normalization and ReLU activation are performed after each convolution to obtain the feature map f3; the feature maps f2 and f3 are sent to a SE module to perform channel weighting and feature mapping on the feature channels to obtain the feature map f5; The feature map f5 is separated into two feature maps with 64 channels, and the two maps are added item by item and then output through a 1×1 convolution layer to obtain the feature map f6. The feature map f3 is obtained by a 3×3 hole convolution layer, a 3×3 convolution layer, batch normalization and ReLU activation to obtain the feature map f4. The feature maps f4 and f6 are added item by item to achieve feature fusion, and the output B is generated after passing through three consecutive 2×2 convolution layers. The output B is further processed to obtain three tensors of the output part of the network: a feature tensor X with a size of W×H×C is obtained through l2 normalization, where C represents the number of channels; 2 The element is squared and 1×1 convolved, and the output is converted into a probability distribution through the Softmax function to obtain a repeatability tensor S of size W×H×1 and a reliability tensor R of size W×H×1, where l2 normalization refers to normalizing the Euclidean norm of each feature vector, x 2 Element-wise squaring means squaring each element in the matrix; The key features are screened in the feature tensor through the reliability tensor and the repeatability tensor to obtain the feature descriptors and key point positions of the key depth features, including: The sparse, repeatable, and highly discriminative key deep features are selected from the feature tensor X by using the repeatability S and the reliability R, and the local maximum of the repeatability S is found and checked whether it exceeds the corresponding reliability R; Through the above steps, the input image to be detected I and the suspected tampering source image I' are used to obtain the feature descriptor of the image to be detected containing M key depth features. and its coordinate position P, the feature descriptor of the suspected tampered source image contains N key depth features and its coordinate position P'.

2. The constrained image stitching detection method based on key depth feature matching according to claim 1 is characterized in that: The calculation formula of the comprehensive features corresponding to the image to be detected is: in, (1) F i Represents the comprehensive features of a key point i in the image to be detected, The feature descriptor representing the key depth feature corresponding to the image to be detected I, P i Indicates the position corresponding to the key point i, MLP en represents a multi-layer perceptron encoder, i = 1, 2, ... M, M represents the number of key feature points of the image to be detected; The calculation formula for the comprehensive features corresponding to the suspected tampered source image is: in, (1) F' j Represents the comprehensive features of a key point j in the suspected tampered source image, The feature descriptor representing the key depth features corresponding to the suspected tampered source image I', P' j Represents the position corresponding to the key point j, j = 1, 2, ... N, N represents the number of key feature points of the suspected tampered source image.

3. The constrained image stitching detection method based on key depth feature matching according to claim 2 is characterized in that: Based on the comprehensive features, the feature map of the matching neural network is calculated layer by layer through attention weighting, and the final enhanced feature map is output, including: The initial input feature matrix of the image to be detected is (1) F, the initial input feature matrix of the suspected tampered source image is (1) F' query, query (l) Q, key (l) K, value (l) V is the matrix required for the attention calculation of the lth layer of the image to be detected, (l) Q', (l) K', (l) V' is the matrix required for the attention calculation of the lth layer of the suspected tampered source image, (l) F and (l) F' represents the intermediate representation matrix of the lth iteration, l ≥ 1, and the calculation method is selected according to the number of iteration layers l; If l is an odd number, (l) F and (l) F' performs self-attention calculations respectively and calculates (l) Q, (l) K, (l) V and (l) Q', (l) K', (l) V' matrix, according to (l) Q and (l) K gets the self-attention weight (l) A self ,according to (l) Q' and (l) K'get (l) A' self , the attention value is calculated by the attention weight (l) O and (l) O', the output updated features are (l+1) F= (l) F+ (l) MLP( (l) F|| (l) O) and (l+1) F'= (l) F'+ (l) MLP( (l) F'|| (l) O'); (l) MLP represents the multi-layer perceptron of the lth layer, (·||·) represents splicing along the channel direction; If l is even, (l) F and (l) F' performs two-way mutual attention calculation and calculates (l) Q, (l) K, (l) V and (l) Q', (l) K', (l) V' matrix, according to (l) Q and (l) K gets the mutual attention weight (l) A cross ,according to (l) Q' and (l) K' gets the mutual attention weight (l) A' cross , calculated by mutual attention weights (l) O and (l) O', the output updated features are (l+1) F= (l) F+ (l) MLP( (l) F|| (l) O) and F' (l+1) = (l) F'+ (l) MLP( (l) F'|| (l) O'); l starts from 1 until the iteration reaches the required number of network layers L, and finally outputs the feature matrix of the image to be detected (L) F and the feature matrix of the suspected tampered source image (L) F'.

4. The constrained image stitching detection method based on key depth feature matching according to claim 3 is characterized in that: The score matrix calculation formula is: S=< (L) F, (l) F'> Where S represents the score matrix and <·,·> represents the inner product operation.

5. The constrained image stitching detection method based on key depth feature matching according to claim 4 is characterized in that: Assume that the image to be detected and the suspected tampered source image have M and N key points respectively, let the score matrix S∈R M×N Represents the matching scores between all possible key point pairs, expanding the score matrix for 6. The constrained image stitching detection method based on key depth feature matching according to claim 5 is characterized in that: By iteratively normalizing the rows and columns of the extended score matrix, the allocation matrix is ​​obtained, which specifically includes: Convert the expanded score matrix to a non-negative matrix using an exponential function ∈ is the regularization parameter, Introducing vectors (1) u and vector (1) v, initialization (1) u is a full 1 column vector of length M+1, initialized (1) v is a column vector of length N+1 containing all 1s. (t) u represents the vector u of the tth iteration, (t) v represents the vector v of the tth iteration, (t+1) u and (t+1) v represents the vector u and vector v obtained in the next round of iteration, t≥1; the target row margin vector r is a full-one column vector of length M+1, and the target column margin vector c is a full-one column vector of length N+1; At the beginning of the iteration, calculate Where × represents matrix multiplication, Indicates that the vector is used in the tth iteration (t) v pair matrix The result after scaling is Perform row normalization; calculate Where T represents transpose, Indicates that the vector is used in the tth iteration (t) u pair matrix The result after scaling is Perform column normalization; The above two steps of normalization are performed alternately until the vectors u and v converge or the preset number of iterations is reached, and the matrix is ​​obtained. Where diag(·) means generating a diagonal matrix from a vector, that is, the main diagonal elements of the generated matrix are the original vector values, and the rest are 0; From the matrix Remove the extended rows and columns to obtain the allocation matrix 7. A computer device, characterized in that: include: A memory and a processor, wherein a computer program that can be run on the processor is stored in the memory, and when the processor executes the computer program, the steps of any one of the methods of claims 1 to 6 are implemented.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of any one of the methods of claims 1 to 6.

Images