A method of identifying a compression image source camera

Abstract

The application discloses a compressed image source camera recognition method, comprising the following steps: obtaining noise residuals of a plurality of reference compressed images; preprocessing the noise residuals of the plurality of reference compressed images to obtain reference sensor pattern noise; obtaining noise residuals of a test compressed image; preprocessing the noise residuals of the test compressed image to obtain test sensor pattern noise; using peak correlation energy to obtain the correlation between the test sensor pattern noise and the reference sensor pattern noise, and judging the shooting source of the test compressed image through the correlation. The application can extract more complete sensor pattern noise from the compressed image, effectively improve the recognition effect of the compressed image, and has strong robustness in the recognition of compressed images with small resolution.

Description

Technical Field

[0001] This invention relates to the technical field of image forensics, and in particular to a method for identifying compressed image source cameras. Background Technology

[0002] In the information age, the scale of image sharing on social media platforms is expanding rapidly, and multimedia forensics is gaining increasing attention. Among these methods, source camera identification can accurately determine the camera from which an image originated, helping investigators verify the authenticity, integrity, and source of an image. This provides strong support for criminal investigations, intellectual property disputes, and other legal procedures. Therefore, source camera identification has become an indispensable tool in the field of digital forensics, enabling forensic experts to trace the source of shared images across the vast internet. Images are compressed, sometimes multiple times, during transmission on social media; therefore, identifying the source of compressed images has become an important research topic in multimedia information security forensics in recent years.

[0003] Most existing source camera identification methods are based on the inherent traces left in the image by the image acquisition hardware and image processing software, including erroneous pixel information - bad pixels, such as: Geradts ZJ, Bijhold J, Kieft M, et al. Methods for identification of images acquired with digital cameras [C] / / Enabling technologies for law enforcement and security. SPIE, 2001, 4232: 505-512 (Geradts Z J, Bijhold J, Kieft M. Methods for identification of images acquired with digital cameras [C] / / Enabling technologies for law enforcement and security. SPIE, 2001, 4232: 505-512); color filter array interpolation algorithm blurring effect, such as Swaminathan A, Wu M, Liu K JR. Nonintrusive component forensics of visual sensors using output images [J]. IEEE Transactions on Information Forensics and Security, 2007, 2(1): 91-106. (Swaminathan A, Wu M, Liu KJ R. Non-invasive component forensics of visual sensors using output images [J]. IEEE Transactions on Information Forensics and Security, 2007, 2(1): 91-106.); Blocking effect of JPEG compression algorithm, such as Alles EJ, Geradts ZJMH, Veenman CJ. Source camera identification for heavily jpeg compressed low resolution stillimages [J]. Journal of forensic sciences, 2009, 54(3): 628-638.; Lens distortion, such as San Choi K, Lam EY, Wong K KY.Source camera identification using footprints from lens aberration[C] / / Digital photography II.SPIE,2006,6069:172-179.(San Choi K,Lam EY,Wong KK Y. Source camera identification using footprints from lens aberration[C] / / Digital photography II.Spie,2006,6069:172-179.); Changes in image statistical characteristics caused by lens image distortion, such as: Dirik AE,Sencar HT,Memon N. Digital singlelens reflex camera identification from traces of sensor dust[J].IEEE Transactions on Information Forensics and Security,2008,3(3):539-552.(Dirik AE,Sencar HT,Memon N. Identification of digital single-lens reflex cameras from traces of sensor dust[J].IEEE Transactions on Information Forensics and Security,2008,3(3):539-552); Mode noise of imaging sensors, such as Lukas J,Fridrich J,Goljan M.Digital camera identification from sensor pattern noise[J].IEEETransactions on Information Forensics&Security,2006,1(2):205-214.

[0004] (Lukas J, Fridrich J, Goljan M. Identifying digital cameras from sensor pattern noise [J]. IEEE Transactions on Information Forensics & Security, 2006, 1(2): 205-214.) etc.

[0005] An effective digital image source detection algorithm must possess strong robustness and a high detection rate. However, factors such as CFA interpolation features and sensor dust introduce instability, necessitating the development of a stable feature for identification. Following in-depth research, some scholars have proposed a digital image source detection method based on sensor pattern noise. This method utilizes sensor pattern noise as a unique fingerprint of the camera, enabling accurate source detection under various conditions.

[0006] As a crucial component of camera imaging, sensors, due to defects in materials and manufacturing processes, produce unique noise and artifacts. Even cameras of the same model can exhibit different noise artifacts. This unique noise artifact is known as sensor pattern noise, and thus serves as an effective fingerprint for camera identification. The main component of sensor pattern noise is non-uniform light response noise, also referred to as the sensor pattern. Therefore, sensor patterns can be used as a camera fingerprint for source detection of unknown images and images. By analyzing and extracting sensor pattern information, we can determine the source camera of an image or image, which is of great significance for forensic and identification applications, providing effective technical support for image and image source detection.

[0007] With the exploration of source camera identification based on sensor patterns, some scholars have proposed a series of effective sensor pattern extraction algorithms, such as: [7] Lukas J, Fridrich J, Goljan M. Digital camera identification from sensor pattern noise[J]. IEEE Transactions on Information Forensics & Security, 2006, 1(2): 205-214.(Lukas J, Fridrich J, Goljan M. Digital camera identification from sensor pattern noise[J]. Journal of Information Forensics and Security, 2006, 1(2): 205-214) proposed a maximum likelihood estimator (MLE) to extract the multiplicative factor of sensor pattern, and proposed an enhancement algorithm using zero mean (ZM) operation and Wiener filter (WF) in the Fourier domain to suppress non-sensor pattern components such as CFA interpolation artifacts; such as: X.Kang, Y.Li, Z.Qu and J.Huang, "Enhancing Source Camera Identification Performance With "A Camera Reference Phase Sensor Pattern Noise," in IEEE Transactions on Information Forensics and Security, vol.7, no.2, pp.393-402, April 2012. (X.Kang, Y.Li, Z.Qu and J.Huang. "Enhancing Source Camera Recognition Performance with Camera Reference Phase Sensor Pattern Noise," IEEE Transactions on Information Forensics and Security, Vol.7, No.7.)2. Pages 393-402, April 2012), proposed using the phase information of reference noise to construct fingerprints; for example: X. Lin and C.-T. Li, "Preprocessing Reference Sensor Pattern Noise via Spectrum Equalization," in IEEE Transactions on InformationForensics and Security, vol.11, no.1, pp.126-140, Jan.2016. (X. Lin and C.-T. Li, "Preprocessing Reference Sensor Pattern Noise via Spectrum Equalization," IEEE Transactions on Information Forensics and Security, Vol.11, No.11, pp.126-140, January 2016), proposed an algorithm for spectrum amplitude equalization to correct local undesirable frequency points; for example: Bhupendra G, Mayank T. Improving Performance of Source Camera Identification By Suppressing Peaks and Eliminating Low-Frequency Defects of Reference SPN[J]. IEEE Signal Processing Letters, 2018, PP(9):1340-1343.(Bhupendra G, Mayank T. Improving source camera identification performance by suppressing peaks and eliminating low-frequency defects using reference SPN[J]. Journal of Signal Processing, 2018, PP(9):1340-1343), proposes a method to suppress low-frequency noise by adding discrete cosine transform on the basis of spectrum equalization; another example: Zeng H, Wan Y, Deng K, et al. Source Camera Identification With Dual-Tree Complex Wavelet Transform[J]. IEEE Access, 2020, 8:18874-18883.(Zeng H, Wan Y, Deng K et al. Source camera identification based on dual-tree complex wavelet transform[J].Journal of Electronic Engineering, 2020, (8): 18874-18883), proposed a spatial adaptive shrinkage algorithm based on dual-tree complex wavelet to extract noise, and proposed a boundary symmetric expansion method to reduce boundary effects; another example: Taspinar S, Mohanty M, Memon N. Camera fingerprint extraction via spatial domain averaged frames[J]. IEEE Transactions on Information Forensics and Security, 2020, 15: 3270-3282. (Taspinar S, Mohanty M, Memon N. Camera fingerprint extraction based on spatial domain averaged frames[J]. Journal of Information Forensics and Security, 2020, 15(5): 370-3282) proposed an algorithm for extracting camera fingerprints based on spatial domain averaged frames, which can greatly speed up the algorithm's running speed; another example: Su Kaiqing, Nili Tian, ​​and Qing Pan. "Multimedia source identification using an improved weight photoresponse non-uniformity noise extraction model in short compressed videos." Forensic Science International: Digital Investigation 42(2022):301473.(SuKaiqing, Nili Tian, ​​and Qing Pan. Identification of Multimedia Sources in Short Compressed Images Using an Improved Weighted Optical Response Non-uniformity Noise Extraction Model. International Forensic Science: Digital Survey 42(2022):301473), proposes an improved dual-tree complex wavelet filtering model and a weighted maximum likelihood estimation algorithm; another example is Timmerman D, Bennabhaktula S, Alegre E, et al. Video camera identification from sensor pattern noise with a constrained convnet[J]. arXiv preprint arXiv:2012.06277,2020.(Timmerman D, Bennabhaktula S, Alegre E, et al. Camera Image Recognition Based on Constrained Convolutional Neural Network[J]. arXiv preprint:2012.06277,2020.)One example is Hui C, Jiang F, Liu S, et al. Source Camera Identification with Multiscale FeatureFusion Network [C] / / 2022IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2022: 1-6. (Hui C, Jiang F, Liu S, et al. Source Camera Identification Based on Multiscale Feature Fusion Network [J] / / 2012IEEE Conference on Multimedia and Information Technology (ICME). IEEE, 2022: 1-6.) This paper proposes a multiscale fusion network to extract sensor pattern information at different scales, making the camera fingerprint contain richer sensor pattern information. For example: Hosseini, MDM, Goljan, M., 2019. Camera identification from HDR images. In: Proceedings of the ACM Workshop on Information Hiding and Multimedia Security, IH&MMSec 2019, Paris, France, July 3-5, 2019, pp. 69-76. (Hosseini, MDM, Goljan, M. Camera identification from HDR images. In: Proceedings of the ACM Workshop on Information Hiding and Multimedia Security, IH&MMSec 2019, Paris, France, July 3-5, 2019, pp. 69-76.) This paper proposes a method to solve high dynamic range image recognition by matching overlapping blocks with adjacent blocks corresponding to camera fingerprints extracted from standard dynamic range images. Another example: Altinisik E, Sencar HT. Source camera verification for strongly stabilized videos[J]. IEEE Transactions on Information Forensics and Security, 2020, 16: 643-657. (Altinisik E, Sencar H T. Verification of cameras with strong stable image sources [J]. Information Forensics and Security, 2020, 16(6): 643-657.) proposes to use a triple grid search algorithm to solve the inversion of the spatial transformation of stable images, thereby reducing the computational load of matching.Most existing algorithms can improve the quality of images and sensor pattern noise within them. However, when it comes to images that have been heavily or repeatedly compressed and distributed over the internet, the original information of the image is significantly destroyed, and sensor pattern noise information is also greatly attenuated. This makes it difficult for existing image extraction algorithms to extract relatively complete sensor pattern noise from compressed images.

[0008] For the reasons mentioned above, in order to extract more complete sensor pattern noise from compressed images and improve the source camera recognition performance of compressed images, it is necessary to explore an effective source camera recognition method for compressed images. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for camera recognition of compressed image sources. This method can extract more complete sensor pattern noise from compressed images, effectively improve the recognition effect of compressed images, and also has strong robustness in the recognition of compressed images with lower resolution.

[0010] To achieve the above objectives, the technical solution provided by this invention is as follows:

[0011] A method for camera recognition of compressed image sources includes:

[0012] Calculate the noise residuals of multiple reference compressed images;

[0013] The noise residuals of multiple reference compressed images are preprocessed to obtain the reference sensor mode noise;

[0014] Calculate the noise residual of the test compressed image;

[0015] The noise residual of the test compressed image is preprocessed to obtain the test sensor mode noise;

[0016] The peak correlation energy is used to determine the correlation between the sensor mode noise of the test compressed image and the sensor mode noise of the reference compressed image, and the source of the test compressed image is determined by the correlation.

[0017] Furthermore, the noise residuals of multiple reference compressed images and the noise residuals of the test compressed image are calculated, including:

[0018] Each image is expanded by multiple pixel-level symmetrical boundaries, and then the expanded image is subjected to multi-layer dual-tree complex wavelet transform to obtain the corresponding high-frequency sub-band and low-frequency sub-band.

[0019] The Wiener filter is applied to filter and estimate all high-frequency subbands to obtain the filtered high-frequency subbands; the dual-tree complex wavelet inverse transform is performed on the filtered high-frequency subbands and low-frequency subbands to obtain the denoised image; the noise residual of each image is obtained by subtracting the images before and after denoising.

[0020] Furthermore, the noise residuals of multiple reference compressed images are preprocessed to obtain the reference sensor mode noise, including:

[0021] Zero-mean processing is applied to the noise residuals of each reference compressed image to remove CFA interpolation artifacts;

[0022] Two-dimensional wavelet transform is performed on the noise residual of the reference compressed image after zero-mean processing to obtain three high-frequency components and one low-frequency approximate component. The highest frequency diagonal component is set to zero, and two-dimensional wavelet inverse transform is performed on the high-frequency and low-frequency components.

[0023] Fourier transform is performed on the noise residual of the reference compressed image after zero-mean and diagonal high-frequency components are zeroed, the zero-frequency components are moved to the center of the spectrum, and the amplitude of the spectrum is smoothed by additive semi-quadratic iterative least squares smoothing to obtain a preliminary smoothed frequency domain signal.

[0024] The difference between the spectral amplitudes before and after additive semi-quadratic iterative least squares smoothing is called the spectral residual. It is assumed that the residual still contains some sensor mode information. The spectral residual is then subjected to iterative least squares processing again. The two adjacent smoothed spectral residuals are iteratively smoothed again in the next iteration. After several iterations, the smoothed spectra are superimposed, and the zero-frequency component is shifted back to zero. The inverse Fourier transform of the frequency domain signal is then performed to obtain the preprocessed noise.

[0025] The multiplicative factor of the sensor mode is calculated using maximum likelihood estimation for the N noise residuals after preprocessing, and then multiplied by the filtered image and the mean value is taken to refer to the sensor mode noise.

[0026] Furthermore, the noise residual of the test compressed image is preprocessed to obtain the test sensor mode noise, including:

[0027] Zero-mean processing is applied to the noise residuals of the test compressed image to remove CFA interpolation artifacts;

[0028] Two-dimensional wavelet transform is performed on the noise residual of the test compressed image after zero-mean processing to obtain three high-frequency components and one low-frequency approximate component. The highest frequency diagonal component is set to zero, and two-dimensional wavelet inverse transform is performed on the high-frequency and low-frequency components.

[0029] Fourier transform is performed on the noise residual of the test compressed image after zero-mean and diagonal high-frequency components are zeroed, the zero-frequency components are moved to the center of the spectrum, and the amplitude of the spectrum is smoothed by additive semi-quadratic iterative least squares smoothing to obtain a preliminary smoothed frequency domain signal.

[0030] The difference between the spectral amplitudes before and after additive semi-quadratic iterative least squares smoothing is called the spectral residual. It is assumed that the residual still contains some sensor mode information. The spectral residual is then subjected to iterative least squares processing again. The two adjacent smoothed spectral residuals are iteratively smoothed again in the next iteration. After several iterations, the smoothed spectra are superimposed, and the zero-frequency component is shifted back to zero. The frequency domain signal is then subjected to inverse Fourier transform to obtain the pre-processed noise. This pre-processed noise is the sensor mode noise of the test compressed image.

[0031] Furthermore, Wiener filters are applied to perform filtering estimations on all high-frequency subbands, including:

[0032] For each image, a two-path filter bank with a binary tree structure is used to decompose and reconstruct the signal. The first tree generates the real part, and the second tree generates the imaginary part. The one-dimensional data formed by each row of each tree is decomposed into one-dimensional wavelet data twice to obtain three high-frequency components and one low-frequency component. Then, the one-dimensional data formed by each column of the high-frequency and low-frequency components is decomposed into one-dimensional wavelet data twice. Each high-frequency sub-band is divided into real and imaginary parts, and multiple high-frequency components are obtained at each level.

[0033] Wiener filtering is applied to each high-frequency subband, as shown below:

[0034]

[0035] W in Represented as wavelet coefficients before filtering, Wo ut Represented as filtered wavelet coefficients, subband local variance estimation

[0036]

[0037] Wint(i,j) is a square window with a side length of h centered at (i,j); max() represents the maximum value of the expression, and min() represents the minimum value of the expression.

[0038] Furthermore, a two-dimensional wavelet transform is performed on the noise residual of the zero-mean processed reference compressed image to obtain the highest frequency component of the signal, which is also the diagonal high-frequency component D, the vertical high-frequency component V, the horizontal high-frequency component H, and the low-frequency component A.

[0039] [D,V,H,A]=DWT2(W ZM (3)

[0040] After zeroing out the diagonal high-frequency components, the signal is reconstructed using inverse wavelet transform of the remaining three components. The noise residual of the reference compressed image after zero-mean and zeroing out the diagonal high-frequency components is then analyzed.

[0041]

[0042] Furthermore, the noise residual W of the reference compressed image after zeroing the mean and diagonal high-frequency components is... ZM-RD Perform a Fourier transform to shift the corresponding zero-frequency component to the corresponding frequency center, thus obtaining the frequency domain signal of the noise residual.

[0043]

[0044] Where Shift represents moving the zero-frequency component to the center of the spectrum, and abs represents taking the modulus. Represents the Fourier transform; for frequency domain signals Smoothing for global optimization

[0045]

[0046] Among them, f s It is a spectral signal with the zero-frequency component shifted to the center. The amplitude at position s, l s λ represents the amplitude of the smoothed spectral signal at position s, λ is the parameter for adjusting the smoothing intensity, and s represents the position of the spectral point. This represents the gradient of a frequency point along the x-axis or y-axis. The penalty function φ W for:

[0047]

[0048] The solution is obtained using an iterative least squares method based on semi-quadratic optimization.

[0049]

[0050] And Fourier transform is used to improve computational efficiency.

[0051]

[0052] express The complex conjugate, It is the Fourier transform of the delta function, and the numerator uses... Simplify; after m iterations, obtain the smoothed result. Signal amplitude l m Then, the zero-frequency component is shifted back to zero and an inverse Fourier transform is performed to obtain a preliminarily smoothed signal.

[0053] Furthermore, in the process of obtaining the reference sensor mode noise, the cyclic spectrum recovery algorithm is expressed as:

[0054] Record the noise residual W of the reference compressed image after zero-mean and zeroing of diagonal high-frequency components. ZM-RD Phase information

[0055]

[0056] The process of spectrum recovery in the j-th iteration is as follows: the spectrum F in the j-th iteration... j The smoothed amplitude S is obtained after RILS processing. j :

[0057] S j =RILS(F j ),j=1,2,3… (11)

[0058] The spectral residual F j With S j The spectral residual F obtained by subtraction j+1 j = 1, 2, 3, ..., the new spectral residual F j+1 Then it undergoes RILS processing and enters the next loop:

[0059] F j+1 =F j -S j ,j=1,2,3,… (12)

[0060] After several iterations, the smoother spectral component S from each RILS processing is obtained. j Superimposed, preprocessed noise signal W p S generated by multiple loops j superposition and with The inverse Fourier expression of multiplication,

[0061]

[0062] real indicates taking the real part of the complex number. This represents the inverse Fourier transform.

[0063] Furthermore, the peak correlation energy is used to determine the correlation between the sensor mode noise of the test compressed image and the sensor mode noise of the reference compressed image. If the peak correlation energy exceeds a given threshold, it is determined that the source of the test compressed image is the camera of the reference image; otherwise, the source of the test compressed image is not the camera.

[0064] Peak-related energy is expressed as:

[0065]

[0066] Where sign() is the sign function, C RQ(a,b) represents the two-dimensional cyclic cross-correlation between the sensor mode noise R of the reference compressed image and the sensor mode noise Q of the test compressed image, β is the region around (0,0), |β| is the area of ​​the β region, and MN is the dimensional product of the matched sensor mode noise.

[0067] Compared with existing technologies, the principles and advantages of this solution are as follows:

[0068] 1) Compared to wavelets, dual-tree complex wavelets exhibit shift invariance and better direction selectivity. The DWT, with two parallel trees, decomposes the signal; one tree represents the real part of the transform, and the other the imaginary part, maintaining the imaginary part sampling position within the middle of the real part to achieve information complementarity. Each tree provides four directions, with eight directional components per layer, thus better representing the image's detailed texture and reducing artifacts caused by denoising.

[0069] 2) A diagonal high-frequency cancellation algorithm is introduced, which uses a one-layer two-dimensional Haar wavelet transform to set the diagonal components to zero, effectively reducing the existing diagonal artifacts and the interference of the highest frequency thermal noise.

[0070] 3) Spectral smoothing based on additive semi-quadratic optimization, compared to spectral equalization algorithms, can perform global optimization smoothing and has edge-preserving capabilities that can protect reasonable local peaks, thereby reducing the loss of sensor pattern information caused by smoothing. Iterative least squares method is used to solve the optimization model, and Fourier transform is used to accelerate the solution. Only a few iterations of Fourier transform and inverse Fourier transform are needed to achieve effective spectral artifact removal.

[0071] 4) An iterative recycling strategy is used to collect sensor mode information of different intensities in turn. The spectrum before and after fall least squares smoothing is subtracted in turn, and this difference is substituted into the next spectrum smoothing algorithm. This process is repeated multiple times, and finally the smoothed terms are superimposed and inverse Fourier transform is performed.

[0072] 5) Compared with existing sensor pattern noise extraction algorithms, this scheme can reduce the interference caused by sensor pattern noise mixed in with image edges. At the same time, it can effectively reduce the loss of sensor pattern information while suppressing non-unique artifacts. It can obtain noise with more complete sensor pattern information. Therefore, this scheme has better performance and robustness in small-sized image recognition and JPEG compressed image recognition. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the services required in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0074] Figure 1 This is a flowchart illustrating the principle of a compressed image source camera recognition method according to the present invention.

[0075] Figure 2 This is a flowchart illustrating the principle of obtaining the test sensor mode noise based on the noise residual of the preprocessed test compressed image in a compressed image source camera identification method of the present invention. Detailed Implementation

[0076] The present invention will be further described below with reference to specific embodiments:

[0077] like Figure 1 As shown in this embodiment, a compressed image source camera recognition method includes:

[0078] S1. Calculate the noise residuals of multiple reference compressed images. The process includes:

[0079] Each reference compressed image is expanded by 8 pixels of symmetrical boundary, and then the expanded image is subjected to 4-layer dual-tree complex wavelet transform to obtain the corresponding high-frequency sub-band and low-frequency sub-band.

[0080] The Wiener filter is applied to filter and estimate all high-frequency subbands to obtain the filtered high-frequency subbands; the dual-tree complex wavelet inverse transform is performed on the filtered high-frequency and low-frequency subbands to obtain the denoised image; the noise residual of each reference compressed image is obtained by subtracting the images before and after denoising.

[0081] In the above, the Wiener filter is applied to perform filtering estimation on all high-frequency subbands, including:

[0082] For each image, a two-path filter bank with a binary tree structure is used to decompose and reconstruct the signal. The first tree generates the real part, and the second tree generates the imaginary part. The one-dimensional data formed by each row of each tree is decomposed into one-dimensional wavelet data twice to obtain three high-frequency components and one low-frequency component. Then, the one-dimensional data formed by each column of the high-frequency and low-frequency components is decomposed into one-dimensional wavelet data twice. Each high-frequency sub-band is divided into real and imaginary parts, and multiple high-frequency components are obtained at each level.

[0083] Wiener filtering is applied to each high-frequency subband, as shown below:

[0084]

[0085] W in W represents the wavelet coefficients before filtering. out Represented as filtered wavelet coefficients, subband local variance estimation

[0086]

[0087] W int (i,j) is a square window with a side length of h centered at (i,j); max() represents the maximum value of the expression, and min() represents the minimum value of the expression.

[0088] S2. Preprocess the noise residuals of multiple reference compressed images to obtain the reference sensor mode noise, including:

[0089] Zero-mean processing is applied to the noise residuals of each reference compressed image to remove CFA interpolation artifacts;

[0090] Two-dimensional wavelet transform is performed on the noise residual of the reference compressed image after zero-mean processing to obtain three high-frequency components and one low-frequency approximate component. The highest frequency diagonal component is set to zero, and two-dimensional wavelet inverse transform is performed on the high-frequency and low-frequency components.

[0091] in,

[0092] A two-dimensional wavelet transform is performed on the noise residual of the zero-mean compressed reference image to obtain the highest frequency component of the signal, which is also the diagonal high-frequency component D, the vertical high-frequency component V, the horizontal high-frequency component H, and the low-frequency component A.

[0093] [D,V,H,A]=DWT2(W ZM (3)

[0094] After zeroing out the diagonal high-frequency components, the signal is reconstructed using inverse wavelet transform of the remaining three components. The noise residual of the reference compressed image after zero-mean and zeroing out the diagonal high-frequency components is then analyzed.

[0095]

[0096] Fourier transform is performed on the noise residual of the reference compressed image after zero-mean and diagonal high-frequency components are zeroed, the zero-frequency components are moved to the center of the spectrum, and the amplitude of the spectrum is smoothed by additive semi-quadratic iterative least squares smoothing to obtain a preliminary smoothed frequency domain signal.

[0097] The difference between the spectral amplitudes before and after additive semi-quadratic iterative least squares smoothing is called the spectral residual. It is assumed that the residual still contains some sensor mode information. The spectral residual is then subjected to iterative least squares processing again. The two adjacent smoothed spectral residuals are iteratively smoothed again in the next iteration. After several iterations, the smoothed spectra are superimposed, and the zero-frequency component is shifted back to zero. The inverse Fourier transform of the frequency domain signal is then performed to obtain the preprocessed noise.

[0098] Among them, the noise residual W of the reference compressed image after zeroing the mean and diagonal high-frequency components is... ZM-RD Perform a Fourier transform to shift the corresponding zero-frequency component to the corresponding frequency center, thus obtaining the frequency domain signal of the noise residual.

[0099]

[0100] Where Shift indicates moving the zero-frequency component to the center of the spectrum, and abs indicates taking the modulus. Represents the Fourier transform; for frequency domain signals Smoothing for global optimization

[0101]

[0102] Among them, f s It is a spectral signal with the zero-frequency component shifted to the center. The amplitude at position s, l s λ represents the amplitude of the smoothed spectral signal at position s, λ is the parameter for adjusting the smoothing intensity, and s represents the position of the spectral point. This represents the gradient of a frequency point along the x-axis or y-axis. The penalty function φ W for:

[0103]

[0104] The solution is obtained using an iterative least squares method based on semi-quadratic optimization.

[0105]

[0106] And Fourier transform is used to improve computational efficiency.

[0107]

[0108] express The complex conjugate, It is the Fourier transform of the delta function, and the numerator uses... Simplify; after m iterations, obtain the smoothed result. Signal amplitude l mThen, the zero-frequency component is shifted back to zero and an inverse Fourier transform is performed to obtain a preliminarily smoothed signal.

[0109] The cyclic spectrum recycling algorithm is expressed as:

[0110] Record the noise residual W of the reference compressed image after zero-mean and zeroing of diagonal high-frequency components. ZM-RD Phase information

[0111]

[0112] The process of spectrum recovery in the j-th iteration is as follows: the spectrum F in the j-th iteration... j The smoothed amplitude S is obtained after RILS processing. j :

[0113] S j =RILS(F j ),j=1,2,3… (11)

[0114] The spectral residual F j With S j The spectral residual F obtained by subtraction j+1 j = 1, 2, 3, ..., the new spectral residual F j+1 Then it undergoes RILS processing and enters the next loop:

[0115] F j+1 =F j -S j ,j=1,2,3,… (12)

[0116] After several iterations, the smoother spectral component S from each RILS processing is obtained. j Superimposed, preprocessed noise signal W p S generated by multiple loops j superposition and with The inverse Fourier expression of multiplication,

[0117]

[0118] real indicates taking the real part of the complex number. This represents the inverse Fourier transform.

[0119] The multiplicative factor of the sensor pattern is calculated using maximum likelihood estimation for the N noise residuals after preprocessing, and then multiplied with the filtered image and the mean value is taken to refer to the sensor pattern noise.

[0120] The calculation formula is as follows:

[0121]

[0122] In the formula, G is the number of reference images used to estimate the sensor mode noise factor K, and N is... z I represents the noise residual of the z-th image. z This represents the z-th reference image.

[0123] S3. Calculate the noise residual of the test compressed image. The process includes:

[0124] The test compressed image is expanded by 8 pixels of symmetrical boundary, and then the expanded image is subjected to 4-layer dual-tree complex wavelet transform to obtain the corresponding high-frequency sub-band and low-frequency sub-band.

[0125] The Wiener filter is applied to filter and estimate all high-frequency subbands to obtain the filtered high-frequency subbands; the dual-tree complex wavelet inverse transform is performed on the filtered high-frequency and low-frequency subbands to obtain the denoised image; the noise residual of the test compressed image is obtained by subtracting the images before and after denoising.

[0126] S4, such as Figure 2 As shown, the noise residual of the test compressed image is preprocessed to obtain the test sensor mode noise, including:

[0127] Zero-mean processing is applied to the noise residuals of the test compressed image to remove CFA interpolation artifacts;

[0128] Two-dimensional wavelet transform is performed on the noise residual of the test compressed image after zero-mean processing to obtain three high-frequency components and one low-frequency approximate component. The highest frequency diagonal component is set to zero, and two-dimensional wavelet inverse transform is performed on the high-frequency and low-frequency components.

[0129] Fourier transform is performed on the noise residual of the test compressed image after zero-mean and diagonal high-frequency components are zeroed, the zero-frequency components are moved to the center of the spectrum, and the amplitude of the spectrum is smoothed by additive semi-quadratic iterative least squares smoothing to obtain a preliminary smoothed frequency domain signal.

[0130] The difference between the spectral amplitudes before and after additive semi-quadratic iterative least squares smoothing is called the spectral residual. It is assumed that the residual still contains some sensor mode information. The spectral residual is then subjected to iterative least squares processing again. The two adjacent smoothed spectral residuals are iteratively smoothed again in the next iteration. After several iterations, the smoothed spectra are superimposed, and the zero-frequency component is shifted back to zero. The frequency domain signal is then subjected to inverse Fourier transform to obtain the pre-processed noise. This pre-processed noise is the sensor mode noise of the test compressed image.

[0131] S5. Use peak correlation energy to determine the correlation between the sensor mode noise of the test compressed image and the sensor mode noise of the reference compressed image. If the peak correlation energy exceeds a given threshold, it is determined that the source of the test compressed image is the camera of the reference image; otherwise, the source of the test compressed image is not the camera.

[0132] Peak-related energy is expressed as:

[0133]

[0134] Where sign() is the sign function, C RQ (a,b) represents the two-dimensional cyclic cross-correlation between the sensor mode noise R of the reference compressed image and the sensor mode noise Q of the test compressed image, β is the region around (0,0), |β| is the area of ​​the β region, and MN is the dimensional product of the matched sensor mode noise.

[0135] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Therefore, any changes made in accordance with the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for original camera recognition of compressed images, characterized in that, include: Calculate the noise residuals of multiple reference compressed images; The noise residuals of multiple reference compressed images are preprocessed to obtain the reference sensor mode noise; Calculate the noise residual of the test compressed image; The noise residual of the test compressed image is preprocessed to obtain the test sensor mode noise; The peak correlation energy is used to determine the correlation between the sensor mode noise of the test compressed image and the sensor mode noise of the reference compressed image, and the source of the test compressed image is determined by the correlation. Preprocessing is performed on the noise residuals of multiple reference compressed images to obtain the reference sensor mode noise, including: Zero-mean processing is applied to the noise residuals of each reference compressed image to remove CFA interpolation artifacts; Two-dimensional wavelet transform is performed on the noise residual of the reference compressed image after zero-mean processing to obtain three high-frequency components and one low-frequency approximate component. The highest frequency diagonal component is set to zero, and two-dimensional wavelet inverse transform is performed on the high-frequency and low-frequency components. Fourier transform is performed on the noise residual of the reference compressed image after zero-mean and diagonal high-frequency components are zeroed, the zero-frequency components are moved to the center of the spectrum, and the amplitude of the spectrum is smoothed by additive semi-quadratic iterative least squares smoothing to obtain a preliminary smoothed frequency domain signal. The difference between the spectral amplitudes before and after additive semi-quadratic iterative least squares smoothing is called the spectral residual. It is assumed that the residual still contains some sensor mode information. The spectral residual is then subjected to iterative least squares processing again. The two adjacent smoothed spectral residuals are iteratively smoothed again in the next iteration. After several iterations, the smoothed spectra are superimposed, and the zero-frequency component is shifted back to zero. The inverse Fourier transform of the frequency domain signal is then performed to obtain the preprocessed noise. The multiplicative factor of the sensor mode is calculated using maximum likelihood estimation for the N noise residuals after preprocessing, and then multiplied by the filtered image and the mean value is taken to refer to the sensor mode noise.

2. The method for original camera recognition of compressed images according to claim 1, characterized in that, Calculate the noise residuals of multiple reference compressed images and the noise residuals of the test compressed image, including: Each image is expanded by multiple pixel-level symmetrical boundaries, and then the expanded image is subjected to multi-layer dual-tree complex wavelet transform to obtain the corresponding high-frequency sub-band and low-frequency sub-band. The Wiener filter is applied to filter and estimate all high-frequency subbands to obtain the filtered high-frequency subbands; the dual-tree complex wavelet inverse transform is performed on the filtered high-frequency subbands and low-frequency subbands to obtain the denoised image; the noise residual of each image is obtained by subtracting the images before and after denoising.

3. The method for original camera recognition of compressed images according to claim 1, characterized in that, The noise residual of the test compressed image is preprocessed to obtain the test sensor mode noise, including: Zero-mean processing is applied to the noise residuals of the test compressed image to remove CFA interpolation artifacts; Two-dimensional wavelet transform is performed on the noise residual of the test compressed image after zero-mean processing to obtain three high-frequency components and one low-frequency approximate component. The highest frequency diagonal component is set to zero, and two-dimensional wavelet inverse transform is performed on the high-frequency and low-frequency components. Fourier transform is performed on the noise residual of the test compressed image after zero-mean and diagonal high-frequency components are zeroed, the zero-frequency components are moved to the center of the spectrum, and the amplitude of the spectrum is smoothed by additive semi-quadratic iterative least squares smoothing to obtain a preliminary smoothed frequency domain signal. The difference between the spectral amplitudes before and after additive semi-quadratic iterative least squares smoothing is called the spectral residual. It is assumed that the residual still contains some sensor mode information. The spectral residual is then subjected to iterative least squares processing again. The two adjacent smoothed spectral residuals are iteratively smoothed again in the next iteration. After several iterations, the smoothed spectra are superimposed, and the zero-frequency component is shifted back to zero. The frequency domain signal is then subjected to inverse Fourier transform to obtain the pre-processed noise. This pre-processed noise is the sensor mode noise of the test compressed image.

4. The method for original camera recognition of compressed images according to claim 2, characterized in that, The Wiener filter is applied to perform filtering estimations on all high-frequency subbands, including: For each image, a two-path filter bank with a binary tree structure is used to decompose and reconstruct the signal. The first tree generates the real part, and the second tree generates the imaginary part. The one-dimensional data formed by each row of each tree is decomposed into one-dimensional wavelet data twice to obtain three high-frequency components and one low-frequency component. Then, the one-dimensional data formed by each column of the high-frequency and low-frequency components is decomposed into one-dimensional wavelet data twice. Each high-frequency sub-band is divided into real and imaginary parts, and multiple high-frequency components are obtained at each level. Wiener filtering is applied to each high-frequency subband, as shown below: （1）； Represented as wavelet coefficients before filtering. Represented as filtered wavelet coefficients, subband local variance estimation , （2）； For A square window with a center point and a side length of h; max() represents the maximum value of the expression, and min() represents the minimum value of the expression.

5. The method for original camera recognition of compressed images according to claim 1, characterized in that, A two-dimensional wavelet transform is performed on the noise residual of the zero-mean compressed reference image to obtain the highest frequency component of the signal, which is also the diagonal high-frequency component D, the vertical high-frequency component V, the horizontal high-frequency component H, and the low-frequency component A. （3）； The diagonal high-frequency components are set to zero, and the remaining three components are used to reconstruct the signal using inverse wavelet transform. The noise residual of the reference compressed image after zero-mean and zeroing of the diagonal high-frequency components is... (4).

6. The method for original camera recognition of compressed images according to claim 5, characterized in that, Noise residual of the reference compressed image after zeroing the mean and diagonal high-frequency components Perform a Fourier transform to shift the corresponding zero-frequency component to the corresponding frequency center, thus obtaining the frequency domain signal of the noise residual. : （5）； Where Shift indicates moving the zero-frequency component to the center of the spectrum. Indicates modulo, Represents the Fourier transform; for frequency domain signals Smoothing for global optimization （6）； in, It is a spectral signal with the zero-frequency component shifted to the center. The magnitude at position s, It is the amplitude at position s of the smoothed spectral signal. This adjusts the smoothing intensity parameter, where 's' represents the position of the spectral point. Represents the gradient of a spectral point along the x-axis or y-axis; penalty function. for: （7）； The solution is obtained using an iterative least squares method based on semi-quadratic optimization. （8）； And Fourier transform is used to improve computational efficiency. （9）； express The complex conjugate, It is the Fourier transform of the delta function, and the numerator uses... Simplify; after m iterations, obtain the smoothed result. signal amplitude Then, the zero-frequency component is shifted back to zero and an inverse Fourier transform is performed to obtain a preliminarily smoothed signal.

7. The method for original camera recognition of compressed images according to claim 6, characterized in that, In the process of obtaining the reference sensor mode noise, the cyclic spectrum recovery algorithm is expressed as: Record the noise residual of the reference compressed image after zero-mean and zeroing of diagonal high-frequency components. Phase information , （10）； The process of recovering the j-th spectrum is as follows: the j-th spectrum... The amplitude is obtained after RILS processing. : （11）； spectral residuals and Spectral residuals obtained by subtraction , = 1,2,3,..., new spectral residuals Then it undergoes RILS processing and enters the next loop: （12）； After several iterations, the smoother spectral components of each RILS processing are obtained. Superimposed, preprocessed noise signal Generated by multiple loops superposition and with The inverse Fourier expression of multiplication, （13）； real indicates taking the real part of the complex number. This represents the inverse Fourier transform.

8. The method for original camera recognition of compressed images according to claim 6, characterized in that, The peak correlation energy is used to determine the correlation between the sensor mode noise of the test compressed image and the sensor mode noise of the reference compressed image. If the peak correlation energy exceeds a given threshold, it is determined that the test compressed image was captured by the same camera as the reference compressed image; otherwise, the test compressed image was not captured by the same camera. Peak-related energy is expressed as: （14）； Where sign() is the sign function, To establish a two-dimensional cyclic cross-correlation between the sensor mode noise R of the reference compressed image and the sensor mode noise Q of the test compressed image, The region surrounding (0, 0). yes The area of ​​the region, where MN is the product of the dimensions of the matched sensor pattern noise.

Images