Camera source identification method based on PRNU

Abstract

The invention discloses a PRNU-based camera source identification method, belongs to the technical field of camera traceability, and aims to solve the technical problem of how to improve the estimation quality of camera fingerprints and improve the accuracy of correlation scores in camera source identification. Comprising the following steps: constructing a camera fingerprint estimation model, introducing a channel attention mechanism into the camera fingerprint estimation model to perform feature extraction and weight distribution on an input image to obtain a weight feature map, extracting a PRNU of the input image through a PRNU extraction network, fusing the weight feature map and the PRNU of the input image, and outputting a camera fingerprint; and a correlation matching model is constructed, and the correlation matching model takes the PRNU of the query image and the camera fingerprint of the target camera as input, and predicts and outputs a correlation score between the query image and the target camera. And for a query image to be traced, outputting a correlation score between the query image and the target camera through the trained correlation matching model.

Description

Technical Field

[0001] The present invention relates to the technical field of camera source tracing, and in particular to a camera source identification method based on PRNU. Background Art

[0002] Image source identification technology based on sensor pattern noise (SPN) has become the main method for camera source identification. The main component of SPN is photo response non-uniformity (PRNU) noise. PRNU noise is unique due to defects in the sensor manufacturing process and has uniqueness and stability. Therefore, it can be used as a device fingerprint for camera source identification and forensic tasks such as image forgery detection and positioning. Unlike other feature-based identification methods, the PRNU-based method can not only identify camera models, but also distinguish different devices of the same model. The core of the PRNU-based camera source identification method lies in the extraction of PRNU, the estimation of camera fingerprints, and the calculation of correlation scores between the two. The PRNU-based camera source identification process is as follows: Figure 1 As shown, denoising filters are usually used to extract PRNU from images or video frames and expressed in the form of a noise matrix. For a set of homologous images, a set of PRNUs is first generated by denoising filtering, and then these PRNUs are processed using methods such as maximum likelihood estimation (MLE) to estimate the camera fingerprint. By calculating the correlation score between the estimated camera fingerprint and the PRNU of the query image and comparing it with the threshold, it can be determined whether the query image comes from the camera.

[0003] The estimation of camera fingerprint in existing camera source identification methods still relies on discrete wavelet transform (DWT) and MLE. How to improve the estimation quality of camera fingerprint and improve the accuracy of correlation score in camera source identification is a technical problem that needs to be solved. Summary of the invention

[0004] The technical task of the present invention is to address the above shortcomings and provide a camera source identification method based on PRNU to solve the technical problem of how to improve the estimation quality of camera fingerprints and improve the accuracy of correlation scores in camera source identification.

[0005] In a first aspect, the present invention provides a camera source identification method based on PRNU, comprising the following steps:

[0006] Construct a camera fingerprint estimation model, which takes the homologous image as the input image, introduces a channel attention mechanism to extract features and assign weights to the input image, obtains a weight feature map, extracts the PRNU of the input image through a PRNU extraction network, and fuses the weight feature map with the PRNU of the input image to output the camera fingerprint of the camera corresponding to the input image;

[0007] Collect homologous images corresponding to multiple camera sources to construct a sample set, perform camera fingerprint estimation based on DWT and MLE methods, use the obtained camera fingerprint as the reference camera fingerprint, train the camera fingerprint estimation model based on the sample set and the reference camera fingerprint, and obtain the trained camera fingerprint estimation model;

[0008] Constructing a correlation matching model, wherein the correlation matching model takes the PRNU of the query image and the camera fingerprint of the target camera as input, is used to calculate the correlation matrix between the PRNU of the query image and the camera fingerprint of the target camera, and calculates the correlation between the query image and the target camera based on the correlation matrix, and outputs a correlation score, wherein the correlation score is used to represent the probability value of the true label, and the true label indicates that the query image and the target camera are homologous;

[0009] Collect homologous images corresponding to multiple camera source cameras and collect multiple query images of known camera sources, extract camera fingerprints of homologous images as camera fingerprint samples through the trained camera fingerprint estimation model, extract PRNU of query images as PRNU samples through the PRNU extraction network, and train the correlation matching model based on the camera fingerprint samples and PRNU samples to obtain a trained correlation matching model;

[0010] For the query image to be traced, the homologous image of the target camera is obtained, the camera fingerprint of the homologous image is extracted through the trained camera fingerprint estimation model, and the PRNU of the query image to be traced is extracted through the PRNU extraction network. With PRNU and camera fingerprint as input, the correlation score between the query image and the target camera is output through the trained correlation matching model.

[0011] Preferably, the camera fingerprint estimation model includes an image preprocessing module, a feature extraction module, a weight generation module, a PRNU feature extraction network and a fusion module;

[0012] The image preprocessing module takes the homologous image as input and is used to preprocess the homologous image to obtain a grayscale image of a predetermined size;

[0013] The feature extraction module introduces a channel attention mechanism, which is used to extract features from the input image using the grayscale image output by the image preprocessing module as the input image;

[0014] The weight generation module takes the feature map output by the feature extraction module as input, and is used to calculate the weight of each feature map and assign a weight to each pixel in the feature map;

[0015] The PRNU feature extraction network uses the homologous image as an input image and is used to extract the PRNU;

[0016] The fusion module takes the weight feature map and the PRNU as input, is used to fuse the weight feature map and the PRNU, and outputs the camera fingerprint;

[0017] Correspondingly, when the camera fingerprint estimation model is trained based on the sample set and the real camera fingerprint, the parameters of the feature extraction module, the weight generation module and the fusion module are optimized through the model training to obtain the trained camera fingerprint estimation model.

[0018] Preferably, the feature extraction module includes a convolution sequence layer, a channel attention module and a compression module.

[0019] The convolution sequence layer is used to extract multi-level image quality features from the input image and output a multi-channel feature map. The convolution sequence layer is followed by a ReLU activation function, which introduces nonlinear characteristics.

[0020] The channel attention module is located at the output end of the convolution sequence layer. It is used to perform global average pooling on the extracted multi-channel feature maps and compress the multi-channel feature maps. The weight of each channel is learned through a combination of two fully connected layers and a nonlinear activation function. The feature map of each channel is multiplied by its weight and the reconstructed feature map is output.

[0021] The compression module takes the reconstructed feature map as input, compresses all extracted feature maps, and obtains a single-channel feature map.

[0022] Preferably, the weight generation module takes the feature map output by the feature extraction module as input, processes the input feature map through a softmax function, and obtains a set of weight feature maps of equal size.

[0023] Preferably, the fusion module is used to obtain the camera fingerprint by performing weighted fusion on the weight feature map and PRNU by multiplying the weight feature map and PRNU point by point.

[0024] Preferably, for the camera fingerprint estimation model, the loss function is a weighted combination of the proportional similarity loss and the numerical loss MSE, and the calculation formula is as follows:

[0025]

[0026] L1=1-corr(A,B),

[0027]

[0028] Among them, A is the set of real camera fingerprints estimated by DWT and MLE methods, B is the set of camera fingerprints predicted by the camera fingerprint estimation model, A i represents the i-th real camera fingerprint in set A, Bi represents the i-th predicted camera fingerprint in set B, and α represents the weight parameter used to balance The influence ratio of loss and mean square error loss MSE.

[0029] Preferably, the correlation matching model includes a data preprocessing module and a correlation score calculation network;

[0030] The data preprocessing module is used to perform the following:

[0031] The PRNU of the camera fingerprint and the query image is trimmed. During the trimming, the camera fingerprint and the query image are respectively used as the target images, and the image blocks of the predetermined pixels are trimmed based on the predetermined coordinate points in the target image to obtain the trimmed camera fingerprint and the trimmed PRNU;

[0032] The cropped camera fingerprint and cropped PRNU are normalized, and the normalization calculation formula is:

[0033]

[0034] Where C is the input matrix, N is the number of all elements in the input matrix, and ∈ represents a constant;

[0035] Calculate the correlation matrix of the normalized camera fingerprint and the normalized PRNU, and normalize the correlation matrix to obtain the normalized correlation matrix. The calculation formula of the correlation matrix is ​​as follows:

[0036]

[0037] Among them, q represents the query image, r represents the camera fingerprint;

[0038] The correlation score calculation network is an improved EfficientNet network after structural reparameterization, and the improved EfficientNet network introduces partial convolution PConv and efficient channel attention mechanism ECA;

[0039] Correspondingly, when the correlation matching model is trained, the model parameters of the correlation score calculation network are optimized, and the trained correlation matching model is obtained based on the data preprocessing module and the trained correlation score calculation network.

[0040] Preferably, before the correlation matching model is trained based on the camera fingerprint sample and the PRNU sample, for each camera fingerprint sample, the correlation score between the camera fingerprint sample and the PRNU sample is calculated by calculating the peak energy correlation ratio between the camera fingerprint sample and each PRNU sample, and for the correlation score below the threshold, the PRNU sample corresponding to the correlation is deleted, and the correlation matching model is trained based on the camera sample and the screened PRNU sample;

[0041] The calculation formula of the peak energy correlation ratio is as follows:

[0042]

[0043] Where ncc(s1,s2,u) represents the normalized cross-correlation value under translation parameters s1, s2 and geometric transformation parameter u, X(i,j) represents the pixel value of the i-th row and j-th column in the camera fingerprint, Y(i+s1,j+s2) represents the pixel value of the i+s1-th row and j+s2-th column in the PRNU of the query image, X represents the camera fingerprint, represents the sample mean of X, Y represents the query image PRNU, represents the sample mean of Y, m represents the number of rows of the image block, and n represents the number of columns of the image block; the translation parameters s1 and s2 are traversed to find the best matching position and maximize ncc. After ncc is maximized, the peak correlation energy ratio PCE(u) between the PNRU of the query image and the camera fingerprint of the target camera is calculated based on the maximized ncc;

[0044] s peak represents the peak position, Indicated by s peak A small area with centered on it, ||·|| represents the two-norm operation.

[0045] As a preference, when training the relevant matching model, binary cross entropy is used as the loss function, which is expressed as:

[0046] BCE(p,y)=-[ylog(p)+(1-y)log(1-p)],

[0047] When y = 1, the true label indicates that the event has occurred, and the probability p predicted by the model is as close to 1 as possible. The loss function penalizes the case where the p value is lower than the threshold. y = 1 indicates that the query image is homologous to the camera fingerprint.

[0048] When y = 0, the true label indicates that the event did not occur, and the probability p predicted by the model is as close to 0 as possible. The loss function penalizes cases where the p value is higher than the threshold. y = 0 indicates that the query image is not homologous to the camera fingerprint.

[0049] The camera source identification method based on PRNU of the present invention has the following advantages:

[0050] 1. A camera fingerprint estimation model was constructed. Based on the excellent feature extraction capability of the PRNU extraction network on a single image, the model introduced a channel attention mechanism to extract the weight feature map and fused the weight feature map with the PRNU extracted by the PNRU extraction network to obtain the camera fingerprint corresponding to the homologous image, thereby improving the accuracy of camera fingerprint estimation.

[0051] 2. For correlation analysis, a correlation matching model was constructed. For this model, the camera fingerprint of the target camera was extracted based on the trained camera fingerprint estimation model, and the PRNU of the query image was extracted through the PRNU extraction network. The correlation between the query image and the homologous image was calculated by calculating the peak energy correlation ratio with the above camera fingerprint and PRNU as input, thereby improving the accuracy of the correlation calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] The present invention is further described below in conjunction with the accompanying drawings.

[0054] Figure 1 A flowchart for camera source identification;

[0055] Figure 2 A flowchart of a camera source identification method based on PRNU is provided in the embodiment;

[0056] Figure 3 It is a structural block diagram of a camera fingerprint estimation model in a camera source identification method based on PRNU in an embodiment;

[0057] Figure 4 It is a structural block diagram of a correlation matching model in a camera source identification method based on PRNU according to an embodiment;

[0058] Figure 5 To implement a PRNU-based camera source recognition method, the accuracy of different methods in different shift conditions is compared;

[0059] Figure 6 The present invention is a comparison of the matching accuracy of different methods under different cropping conditions in a PRNU-based camera source recognition method. DETAILED DESCRIPTION

[0060] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.

[0061] The embodiment of the present invention provides a camera source identification method based on PRNU, which is used to solve the technical problem of how to improve the estimation quality of camera fingerprints and improve the accuracy of correlation scores in camera source identification.

[0062] Example:

[0063] The present invention provides a camera source identification method based on PRNU, such as Figure 2 As shown, it includes five steps: camera fingerprint estimation model construction, camera fingerprint estimation model training, correlation matching model construction, correlation matching model training and camera tracing.

[0064] Step S100: constructing a camera fingerprint estimation model. The constructed camera fingerprint estimation model takes the homologous image as the input image, introduces a channel attention mechanism to perform feature extraction and weight assignment on the input image, obtains a weight feature map, extracts the PRNU of the input image through the PRNU extraction network, and fuses the weight feature map with the PRNU of the input image to output the camera fingerprint of the camera corresponding to the input image.

[0065] In step S100 of this embodiment, the camera fingerprint estimation model includes an image preprocessing module, a feature extraction module, a weight generation module, a PRNU feature extraction network and a fusion module.

[0066] The image preprocessing module takes the homologous image as input and is used to preprocess the homologous image to obtain a grayscale image of a predetermined size.

[0067] As a specific implementation of the image preprocessing module, the image preprocessing module converts the input image into a grayscale image and performs a normalization operation, and outputs an image of size (N, 1, H, W).

[0068] The feature extraction module introduces a channel attention mechanism, which is used to extract features from the input image using the grayscale image output by the image preprocessing module as the input image.

[0069] like Figure 3As shown, the feature extraction module in this embodiment includes a convolution sequence layer, a channel attention module and a compression module. The convolution sequence layer is used to extract multi-level image quality features from the input image and output a multi-channel feature map. A ReLU activation function is configured after the convolution sequence layer, and nonlinear characteristics are introduced through the ReLU activation function; the channel attention module is located at the output end of the convolution sequence layer, and is used to perform a global average pooling operation on the extracted multi-channel feature map and compress the multi-channel feature map. The weight of each channel is learned through a combination of two fully connected layers and a nonlinear activation function, and the feature map of each channel is multiplied by its weight to output a reconstructed feature map; the compression module takes the reconstructed feature map as input, compresses all extracted feature maps, and obtains a single-channel feature map.

[0070] As a specific implementation of the feature extraction module, the sequence of convolutional layers is responsible for extracting multi-level quality features from the grayscale image first. The convolutional layers all use 3×3 convolution kernels to maintain the integrity of local features while reducing the number of parameters and reducing the risk of overfitting of the model. Padding is set to 1 to ensure that the convolution operation does not lose the edge information of the image while maintaining the spatial dimension of the feature map. The first convolutional layer captures basic quality features such as the edge information of the image. As the sequence of convolutional layers deepens, subsequent convolutional layers gradually integrate low-level features and learn more complex quality features. After the ReLU activation function is applied to the convolutional layer, nonlinear characteristics are introduced, allowing the network to learn more complex mapping relationships. The number of output channels of the first two convolutional layers is 32, and the number of output channels of the last four convolutional layers is 64. By gradually increasing the number of channels, multi-dimensional representation of quality features from primary to advanced levels is achieved.

[0071] After the sequence of convolutional layers, the extracted image quality features are adaptively recalibrated using the SE attention module to enhance the model's representation of key features of image quality. This module improves the network's representation capabilities by enabling the network to perform dynamic channel-level feature recalibration. The specific operation is as follows: First, a global average pooling operation is performed on the feature map of input size (N, 64, H, W) to compress the input features to capture global feature statistics. Then, a combination of two fully connected layers and a nonlinear activation function is used to learn the weight of each channel to capture the relationship between channels and obtain the weight of each channel. By learning the weight of each channel, the SE attention module is able to adaptively adjust the feature map. Finally, the feature map of each channel is multiplied by its weight to obtain a reconstructed feature map with an output size of (N, 64, H, W).

[0072] The compression module uses a 1×1 convolution kernel to integrate all the extracted features and outputs a feature map of size (N, 1, H, W). This feature map reflects the quality information of the image, emphasizes the high PRNU information quality part of the image, and suppresses the image information with large noise.

[0073] Based on the above feature extraction module, the key features that reflect the quality of grayscale images are extracted from grayscale images. The input size is (N, 1, H, W), where N represents the number of homologous images used to estimate the camera fingerprint, 1 is the number of input channels, and H and W are the height and width of the input image, respectively. The output of this module is a feature map of the same size containing image quality information.

[0074] The weight generation module takes the feature map output by the feature extraction module as input, and is used to calculate the weight of each feature map and assign a weight to each pixel in the feature map.

[0075] In this embodiment, the weight generation module takes the feature map output by the feature extraction module as input, processes the input feature map through the softmax function, and obtains a set of weight feature maps of equal size.

[0076] The main function of the weight generation module is to generate weight feature maps based on image quality. Its operation process first receives the output feature map from the feature extraction module as input, and then processes these feature maps by applying the softmax function to obtain a set of weight feature maps of equal size. The weight generation module applies different weights to each image and assigns personalized weights to each pixel in the image to more accurately reflect the contribution of each pixel to the image quality.

[0077] The PRNU feature extraction network takes the homologous image as input image and is used to extract PRNU.

[0078] The PRNU feature extraction network of this embodiment is a PRNU extraction network in the prior art, and is a trained network.

[0079] The fusion module takes the weight feature map and PRNU as input, fuses the weight feature map and PRNU, and outputs the camera fingerprint.

[0080] In this embodiment, the fusion module is used to obtain the camera fingerprint by performing weighted fusion on the weight feature map and the PRNU by multiplying the weight feature map and the PRNU point by point.

[0081] The main function of the fusion module is to effectively fuse the PRNU image according to the weight feature map output by the weight generation module to accurately estimate the camera's fingerprint. The input of this module is the weight feature map output by the weight generation module and the PRNU of the homologous image extracted by the PRNU extraction network. The input size of both is (N, 1, H, W). The module first multiplies the weight feature map by the PRNU feature point by point. This operation aims to weight the PRNU features of each image according to the weight feature map. The weighted image PRNU features are then accumulated and superimposed to achieve feature integration. Finally, the module outputs a camera fingerprint of size (1, 1, H, W).

[0082] Correspondingly, when the camera fingerprint estimation model is trained based on the sample set and the real camera fingerprint, the parameters of the feature extraction module, the weight generation module and the fusion module are optimized through the model training to obtain the trained camera fingerprint estimation model.

[0083] Step S200: Camera fingerprint estimation model training: collect homologous images corresponding to multiple camera sources to construct a sample set, perform camera fingerprint estimation based on DWT and MLE methods, use the obtained camera fingerprint as a reference camera fingerprint, and train the camera fingerprint estimation model based on the sample set and the reference camera fingerprint to obtain a trained camera fingerprint estimation model.

[0084] In this embodiment, for the camera fingerprint estimation model, the loss function is a weighted sum of the loss based on proportional similarity and the numerical loss MSE, and the calculation formula is as follows:

[0085]

[0086] L1=1-corr(A,B),

[0087]

[0088] Among them, A is the set of real camera fingerprints estimated by DWT and MLE methods, B is the set of camera fingerprints predicted by the camera fingerprint estimation model, A i represents the i-th real camera fingerprint in set A, B i represents the i-th predicted camera fingerprint in set B, and α represents the weight parameter used to balance The influence ratio of loss and mean square error loss MSE.

[0089] This loss function combines the two and can consider both the global structure and local details of the image. By considering both scale similarity and pixel-level error, the model can learn more robust feature representations during training, thereby improving the model's generalization ability under different data sets and conditions.

[0090] For the camera fingerprint estimation model, the training strategy is as follows: a set of homologous natural images (size (N, 1, H, W) is input into the network, the PRNU of each image is extracted through the PRNU extraction network, and then the set of homologous images and PRNU are input into the camera fingerprint estimation network. This network obtains weights according to image quality and fuses PRNU based on this to estimate the camera fingerprint k1. Then, the set of homologous flat-field images is used to estimate the camera fingerprint based on the DWT and MLE methods to obtain the camera fingerprint k2, which is used as a reference. In practical applications, not all cameras have flat-field images, so we must rely on natural images to estimate the camera fingerprint.

[0091] In the specific implementation of model training, the VISION dataset is selected as the training dataset for the camera fingerprint estimation network. 19 cameras are randomly selected as training devices, and 50 JPEG images are selected for each camera for training. Each image is divided into multiple 40×40 small image blocks. The training samples are camera fingerprint-image groups, each group contains 50 40×40 image blocks and a paired 40×40 camera fingerprint block.

[0092] Step S300: A correlation matching model is constructed. The correlation matching model takes the PRNU of the query image and the camera fingerprint of the target camera as input, is used to calculate the correlation matrix between the PRNU of the query image and the camera fingerprint of the target camera, and calculates the correlation between the query image and the target camera based on the correlation matrix, and outputs a correlation score. The correlation score is used to represent the probability value of the true label, and the true label indicates that the query image and the target camera are homologous.

[0093] The correlation matching model of this embodiment includes a data preprocessing module and a correlation score calculation network.

[0094] The data preprocessing module is used to perform the following:

[0095] (1) The PRNU of the camera fingerprint and the query image is trimmed. During the trimming, the camera fingerprint and the query image are respectively used as the target images, and the image blocks of predetermined pixels are trimmed based on predetermined coordinate points in the target image to obtain the trimmed camera fingerprint and the trimmed PRNU;

[0096] (2) The cropped camera fingerprint and cropped PRNU are normalized. The normalization calculation formula is:

[0097]

[0098] Where C is the input matrix, N is the number of all elements in the input matrix, and ∈ represents a constant;

[0099] (3) Calculate the correlation matrix of the normalized camera fingerprint and the normalized PRNU, and perform a normalization operation on the correlation matrix to obtain a normalized correlation matrix. The calculation formula of the correlation matrix is ​​as follows:

[0100]

[0101] Among them, q represents the query image and r represents the camera fingerprint.

[0102] In this embodiment, the correlation score calculation network is a network model constructed by structurally reparameterizing EfficientNet and introducing partial convolution PConv and efficient channel attention mechanism ECA.

[0103] In this embodiment, partial convolution (PConv) is added to the EfficientNet network to replace the standard convolution operation in the EfficientNet network. In Stage 3 to Stage 7 of the EfficientNet network, the EfficientNet model is configured with a stack of MBConv6. In the improved EfficientNet network of this embodiment, only the first layer of MBConv6 structure is retained, and the MBConv6 structure of the subsequent layers is replaced with PConv. This improved operation effectively reduces the computational complexity of the model by reducing the redundant calculations between feature map channels. At the same time, the original SE attention module in the EfficientNet network is replaced by the efficient channel attention mechanism ECA. The ECA mechanism captures the dependencies between channels through one-dimensional convolution, which improves the efficiency and accuracy of feature representation.

[0104] The structural reparameterization technology of this embodiment is used to decouple the network architecture during training and inference. During the training phase, the network adopts a multi-branch structure to enhance the feature extraction capability of the model. During the inference phase, these multi-branch structures are merged into a single-branch structure, reducing the complexity of the model and the inference time. Structural reparameterization is achieved by converting the parameters of the multi-branch structure into the parameters of the single-branch structure after the training is completed. The specific structural reparameterization process is as follows:

[0105] (1) In the MBConv1 module, four branches are used during training: two 3×3 depth-wise convolution branches, one 1×1 depth-wise convolution branch, and one identity branch. During inference, the above four branches are merged into a single branch structure. Through the superposition characteristics of convolution, multiple convolution operations are fused into one.

[0106] (2) In the MBConv6 module, a multi-branch structure is used during training, including 1×1 convolution and multiple depth-wise convolution branches. The 1×1 convolution is used for channel expansion. During inference, the multi-branch structure used during training is merged into a single branch structure to improve inference efficiency.

[0107] Correspondingly, when the correlation matching model is trained, the model parameters of the correlation score calculation network are optimized, and the trained correlation matching model is obtained based on the data preprocessing module and the trained correlation score calculation network.

[0108] Step S400 correlation matching model training: collect homologous images corresponding to multiple camera sources and collect multiple query images of known camera sources, extract camera fingerprints of homologous images as camera fingerprint samples through the trained camera fingerprint estimation model, extract PRNU of query images as PRNU samples through the PRNU extraction network, and train the correlation matching model based on the camera fingerprint samples and PRNU samples to obtain a trained correlation matching model.

[0109] As a specific implementation of correlation matching model training, before the correlation matching model is trained based on camera fingerprint samples and PRNU samples, for each camera fingerprint sample, the correlation score between the camera fingerprint sample and each PRNU sample is calculated by calculating the peak energy correlation ratio between the camera fingerprint sample and each PRNU sample. For correlation scores below a threshold, the PRNU samples corresponding to the correlation are deleted, and the correlation matching model is trained based on the camera samples and the screened PRNU samples.

[0110] The peak energy correlation ratio is calculated as follows:

[0111]

[0112] Where ncc(s1,s2,u) represents the normalized cross-correlation value under translation parameters s1, s2 and geometric transformation parameter u, X(i,j) represents the pixel value of the i-th row and j-th column in the camera fingerprint, Y(i+s1,j+s2) represents the pixel value of the i+s1-th row and j+s2-th column in the PRNU of the query image, X represents the camera fingerprint, represents the sample mean of X, Y represents the query image PRNU, represents the sample mean of Y, m represents the number of rows of the image block, and n represents the number of columns of the image block; the translation parameters s1 and s2 are traversed to find the best matching position and maximize ncc. After ncc is maximized, the peak correlation energy ratio PCE(u) between the PNRU of the query image and the camera fingerprint of the target camera is calculated based on the maximized ncc;

[0113] s peak represents the peak position, Indicated by s peak A small area with centered on it, ||·|| represents the two-norm operation.

[0114] In this embodiment, the training strategy of the correlation matching model is: Figure 4 As shown in the figure, the PRNU extraction network is used to extract the PRNU of the query image, and the camera fingerprint estimation network is used to estimate the camera fingerprint from N homologous images. The PRNU of the camera fingerprint and the query image are subjected to data preprocessing operations. The specific operations are as follows: first, the PRNU of the camera fingerprint and the query image are cropped, and an image block of a*a and b*b is cropped from the coordinates (x1, y1) and (x2, y2) respectively. We conduct experiments in three cases. In the normal case, a=b, x1=x2, y1=y2, that is, no shift and cropping occurs. In the case of shift, a=b, x2=x1+c, y2=y1+c, c is the shifted pixel. In the case of cropping, a=b+d, d is the cropped pixel. After cropping, the PRNU of the camera fingerprint and the query image are normalized, and after normalization, the correlation is calculated to obtain the correlation matrix, and then the correlation matrix is ​​normalized to obtain the training data.

[0115] Each step of the entire preprocessing process is aimed at improving the quality and effect of signal processing. By normalizing the energy of the noise signal and calculating the correlation matrix, the matrix can be further processed to effectively reduce the interference of noise on the analysis results, enhance the performance of useful signal features, and help accelerate the speed of network convergence while improving accuracy.

[0116] During the model training process, the dataset was constructed as follows: 40 devices in the Dresden dataset were selected for the experiment, and all images taken by these 40 devices were selected. First, the dataset was cleaned, and the peak correlation energy ratio (PCE) was used to calculate the correlation score for the camera fingerprint of the same camera and the PRNU of the query image. The query images with PCE values ​​lower than 30 were removed, and after removal, the training set, validation set, and test set were constructed in a 3:1:1 ratio.

[0117] Based on the above training set, validation set and test set, the cross-correlation ncc is first calculated, and the peak correlation energy ratio of the feature map to be tested and the target camera fingerprint map is calculated based on the maximized ncc.

[0118] Among them, when training the relevant matching model, the binary cross entropy is used as the loss function, which is expressed as:

[0119] BCE(p,y)=-[ylog(p)+(1-y)log(1-p)],

[0120] For a single sample, assuming that its true label y is a binary variable with a value of 0 or 1, and the probability p predicted by the model is between 0 and 1, then the BCE Loss for this sample can be expressed as the calculation formula of BCE(p,y) above.

[0121] When y = 1, the true label indicates that the event has occurred, and the probability p predicted by the model is as close to 1 as possible. The loss function penalizes the case where the p value is lower than the threshold. y = 1 indicates that the query image is homologous to the camera fingerprint.

[0122] When y = 0, the true label indicates that the event did not occur, and the probability p predicted by the model is as close to 0 as possible. The loss function penalizes cases where the p value is higher than the threshold. y = 0 indicates that the query image is not homologous to the camera fingerprint.

[0123] Step S500 camera tracing: for the query image to be traced, obtain the homologous image of the target camera, extract the camera fingerprint of the homologous image through the trained camera fingerprint estimation model, extract the PRNU of the query image to be traced through the PRNU extraction network, take the PRNU and camera fingerprint as input, output the correlation score between the query image and the target camera through the trained correlation matching model, and judge whether the target camera is the camera source of the query image based on the correlation score.

[0124] For the method disclosed in this embodiment, the following experimental verification is given: the PRNU of the test image is correlated with the camera fingerprints of 40 cameras, and the camera with the highest correlation is returned. The accuracy is calculated based on this. The camera fingerprint and PRNU blocks used in this experiment are initially 256×256 pixels. First, the camera fingerprint is estimated using two methods. The camera fingerprint K1 is estimated using the method disclosed in this embodiment, and the camera fingerprint K2 is estimated using the DWT and MLE methods. The camera fingerprint is estimated using the same 50 images. For the extraction of the query image PRNU, three methods, DFPRNU-NET, SPNCNN, and DWT, are used. The correlation is calculated using the correlation matching model trained by PCE and the embodiment. When calculating the correlation using the correlation matching model disclosed in this embodiment, the preprocessing strategy during training is fully followed, and two normalization processes are used.

[0125] Figure 5 In this embodiment, the matching accuracy of different camera source recognition methods under different shift conditions is compared, and x and y in the figure represent the shifted pixels of the horizontal coordinate and the shifted pixels of the vertical coordinate, respectively. Figure 6 In this paper, the matching accuracy of different camera source recognition methods under different cropping conditions is compared. Figure 5 , 6It can be observed that when the correlation calculation method is PCE, the fingerprint estimation performed using the method proposed in this embodiment is the best among all methods using PCE, which indicates that the camera fingerprint estimated using the method proposed in this embodiment has the highest quality and is significantly better than the other methods. When the correlation calculation method is the correlation score calculation network trained in this embodiment, the accuracy of the DWT method and the camera fingerprint estimation method proposed in this embodiment has increased significantly, which indicates the effectiveness of the correlation score calculation network trained in this embodiment, and the effect is significantly better than PCE. By combining the camera fingerprint estimation method proposed in this embodiment and the correlation score calculation network, the method proposed in this embodiment can show the highest accuracy among all methods, which means that the camera source recognition method proposed in this embodiment is better than the existing methods.

[0126] Through the above experiments, it can be obtained that the method of this embodiment uses the PRNU extraction network to train the camera fingerprint estimation network, and at the same time uses the camera fingerprint estimated by the camera fingerprint estimation network for the training of the correlation score calculation network. According to the experimental results, the extraction of the camera fingerprint using this method is obviously better than the existing method, and the accuracy of using the method of this embodiment to train the correlation score calculation network and perform camera source recognition is also better than the original model. It can be seen that the effectiveness of the method proposed in this embodiment. At the same time, this embodiment truly realizes the use of deep learning methods instead of traditional methods in the entire process from PRNU extraction to fingerprint estimation and correlation score calculation, providing new methods and ideas for the field of camera source recognition.

[0127] The method of this embodiment first optimizes the camera fingerprint estimation work. Using the excellent feature extraction ability of the PRNU extraction network on a single image, a camera fingerprint estimation network is trained, and the two are combined to estimate the camera fingerprint. The combined network is used to estimate the camera fingerprint, and the PRNU extraction network is used to extract the PRNU of the query image. At the same time, we optimize the training and testing process of the correlation score calculation network. Using the network we proposed to estimate the camera fingerprint and extract the PRNU, and constructing training samples based on this, the accuracy of the correlation score calculation can be further improved.

[0128] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the means in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A camera source identification method based on PRNU, characterized in that: The steps include: Construct a camera fingerprint estimation model, which takes the homologous image as the input image, introduces a channel attention mechanism to extract features and assign weights to the input image, obtains a weight feature map, extracts the PRNU of the input image through a PRNU extraction network, and fuses the weight feature map with the PRNU of the input image to output the camera fingerprint of the camera corresponding to the input image; Collect homologous images corresponding to multiple camera sources to construct a sample set, perform camera fingerprint estimation based on DWT and MLE methods, use the obtained camera fingerprint as the reference camera fingerprint, train the camera fingerprint estimation model based on the sample set and the reference camera fingerprint, and obtain the trained camera fingerprint estimation model; Constructing a correlation matching model, wherein the correlation matching model takes the PRNU of the query image and the camera fingerprint of the target camera as input, is used to calculate the correlation matrix between the PRNU of the query image and the camera fingerprint of the target camera, and calculates the correlation between the query image and the target camera based on the correlation matrix, and outputs a correlation score, wherein the correlation score is used to represent the probability value of the true label, and the true label indicates that the query image and the target camera are homologous; Collect homologous images corresponding to multiple camera source cameras and collect multiple query images of known camera sources, extract camera fingerprints of homologous images as camera fingerprint samples through the trained camera fingerprint estimation model, extract PRNU of query images as PRNU samples through the PRNU extraction network, and train the correlation matching model based on the camera fingerprint samples and PRNU samples to obtain a trained correlation matching model; For the query image to be traced, the homologous image of the target camera is obtained, the camera fingerprint of the homologous image is extracted through the trained camera fingerprint estimation model, and the PRNU of the query image to be traced is extracted through the PRNU extraction network. With PRNU and camera fingerprint as input, the correlation score between the query image and the target camera is output through the trained correlation matching model.

2. The camera source identification method based on PRNU according to claim 1, characterized in that: The camera fingerprint estimation model includes an image preprocessing module, a feature extraction module, a weight generation module, a PRNU feature extraction network and a fusion module; The image preprocessing module takes the homologous image as input and is used to preprocess the homologous image to obtain a grayscale image of a predetermined size; The feature extraction module introduces a channel attention mechanism, which is used to extract features from the input image using the grayscale image output by the image preprocessing module as the input image; The weight generation module takes the feature map output by the feature extraction module as input, and is used to calculate the weight of each feature map and assign a weight to each pixel in the feature map; The PRNU feature extraction network uses the homologous image as an input image and is used to extract the PRNU; The fusion module takes the weight feature map and the PRNU as input, is used to fuse the weight feature map and the PRNU, and outputs the camera fingerprint; Correspondingly, when the camera fingerprint estimation model is trained based on the sample set and the real camera fingerprint, the parameters of the feature extraction module, the weight generation module and the fusion module are optimized through the model training to obtain the trained camera fingerprint estimation model.

3. The camera source identification method based on PRNU according to claim 2, characterized in that: The feature extraction module includes a convolution sequence layer, a channel attention module and a compression module. The convolution sequence layer is used to extract multi-level image quality features from the input image and output a multi-channel feature map. The convolution sequence layer is followed by a ReLU activation function, which introduces nonlinear characteristics. The channel attention module is located at the output end of the convolution sequence layer. It is used to perform global average pooling on the extracted multi-channel feature maps and compress the multi-channel feature maps. The weight of each channel is learned through a combination of two fully connected layers and a nonlinear activation function. The feature map of each channel is multiplied by its weight and the reconstructed feature map is output. The compression module takes the reconstructed feature map as input, compresses all extracted feature maps, and obtains a single-channel feature map.

4. The camera source identification method based on PRNU according to claim 2, characterized in that: The weight generation module takes the feature map output by the feature extraction module as input, processes the input feature map through the softmax function, and obtains a set of weight feature maps of equal size.

5. The camera source identification method based on PRNU according to claim 2, characterized in that: The fusion module is used to obtain a camera fingerprint by performing weighted fusion on the weight feature map and the PRNU by multiplying the weight feature map and the PRNU point by point.

6. The camera source identification method based on PRNU according to claim 2, characterized in that: For the camera fingerprint estimation model, the loss function is a weighted sum of the proportional similarity-based loss and the numerical loss MSE, and the calculation formula is as follows: L1=1-corr(A,B), Among them, A is the set of real camera fingerprints estimated by DWT and MLE methods, B is the set of camera fingerprints predicted by the camera fingerprint estimation model, A i represents the i-th real camera fingerprint in set A, B i represents the i-th predicted camera fingerprint in set B, and α represents the weight parameter used to balance The influence ratio of loss and mean square error loss MSE.

7. The camera source identification method based on PRNU according to any one of claims 1 to 6, characterized in that: The relevance matching model includes a data preprocessing module and a relevance score calculation network; The data preprocessing module is used to perform the following: The PRNU of the camera fingerprint and the query image is trimmed. During the trimming, the camera fingerprint and the query image are respectively used as the target images, and the image blocks of the predetermined pixels are trimmed based on the predetermined coordinate points in the target image to obtain the trimmed camera fingerprint and the trimmed PRNU; The cropped camera fingerprint and cropped PRNU are normalized, and the normalization calculation formula is: Where C is the input matrix, N is the number of all elements in the input matrix, and ∈ represents a constant; Calculate the correlation matrix of the normalized camera fingerprint and the normalized PRNU, and normalize the correlation matrix to obtain the normalized correlation matrix. The calculation formula of the correlation matrix is ​​as follows: Among them, q represents the query image, r represents the camera fingerprint; The correlation score calculation network is an improved EfficientNet network after structural reparameterization, and the improved EfficientNet network introduces partial convolution PConv and efficient channel attention mechanism ECA; Correspondingly, when the correlation matching model is trained, the model parameters of the correlation score calculation network are optimized, and the trained correlation matching model is obtained based on the data preprocessing module and the trained correlation score calculation network.

8. The camera source identification method based on PRNU according to claim 7, characterized in that: Before the correlation matching model is trained based on the camera fingerprint sample and the PRNU sample, for each camera fingerprint sample, the correlation score between the camera fingerprint sample and the PRNU sample is calculated by calculating the peak energy correlation ratio between the camera fingerprint sample and each PRNU sample, and for the correlation score below the threshold, the PRNU sample corresponding to the correlation is deleted, and the correlation matching model is trained based on the camera sample and the screened PRNU sample; The calculation formula of the peak energy correlation ratio is as follows: Where ncc(s1,s2,u) represents the normalized cross-correlation value under translation parameters s1, s2 and geometric transformation parameter u, X(i,j) represents the pixel value of the i-th row and j-th column in the camera fingerprint, Y(i+s1,j+s2) represents the pixel value of the i+s1-th row and j+s2-th column in the PRNU of the query image, X represents the camera fingerprint, represents the sample mean of X, Y represents the query image PRNU, represents the sample mean of Y, m represents the number of rows of the image block, and n represents the number of columns of the image block; the translation parameters s1 and s2 are traversed to find the best matching position and maximize ncc. After ncc is maximized, the peak correlation energy ratio PCE(u) between the PNRU of the query image and the camera fingerprint of the target camera is calculated based on the maximized ncc; s peak represents the peak position, Indicated by s peak A small area with centered on it, ||·|| represents the two-norm operation.

9. The camera source identification method based on PRNU according to claim 7, characterized in that: When training the correlation matching model, binary cross entropy is used as the loss function, which is expressed as: BCE(p,y)=-[ylog(p)+(1-y)log(1-p)], When y = 1, the true label indicates that the event has occurred, and the probability p predicted by the model is as close to 1 as possible. The loss function penalizes the case where the p value is lower than the threshold. y = 1 indicates that the query image is homologous to the camera fingerprint. When y = 0, the true label indicates that the event did not occur, and the probability p predicted by the model is as close to 0 as possible. The loss function penalizes cases where the p value is higher than the threshold. y = 0 indicates that the query image is not homologous to the camera fingerprint.

Images