Method and system for estimating the offset of non-aligned heavily compressed images based on a two-branch network
By using a dual-branch network-based approach, fine-grained color expansion and second-order difference hierarchical feature extraction are employed to address the issue of insufficient features in JPEG image unaligned recompression offset estimation, thereby improving estimation accuracy.
Patent Information
- Application Number
- CN202311391096.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-25
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2043-10-25
AI Technical Summary
Existing methods suffer from insufficient feature quantity when estimating the unaligned recompression offset of JPEG images, especially for small images, resulting in low estimation accuracy and difficulty in effectively reconstructing the image's operational history.
A dual-branch network-based approach is adopted, which extracts image color channel features through fine-grained color expansion and second-order difference layering. Convolution is used to optimize the difference parameters to separate horizontal and vertical features. A shallow network structure is used for training, and the network parameters are optimized by combining the cross-entropy loss function.
It improves the estimation accuracy of offset in unaligned recompressed images, overcomes the problem of insufficient feature quantity in small-sized images, and achieves higher estimation accuracy.
Smart Images
Figure CN117409064B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of information security technology, specifically to a method and system for estimating the offset of unaligned recompressed images based on a dual-branch network. Background Technology
[0002] JPEG format, due to its advantages such as high transmission speed and small storage space, has become a common storage format in daily life. At the same time, with the development of image tampering technology, it has become easier to tamper with JPEG images. These tampered images may endanger public security. Therefore, JPEG forensics has attracted widespread attention from researchers. Examples include JPEG steganography, JPEG recompression forensics, and JPEG operation history forensics.
[0003] Double compression is a common practice in JPEG recompression. If the discrete cosine transform (DCT) positions of the blocks between the first and second compressions are the same, the compressed image is called an aligned double-compressed image. Otherwise, the compressed image is called an unaligned double-compressed image. For example, a decompressed JPEG image might be re-saved as a JPEG after being tampered with, such as by cropping or partial image replacement. In this case, two JPEG compressions occur: one before and one after the tampering, and the DCT positions of the blocks between the two compressions are different. Figure 5 This demonstrates a scenario of unaligned recompression.
[0004] To reconstruct the operation history of a JPEG image, it is insufficient to simply determine whether the image is an unaligned double-compressed image. The offset between the DCT grids between the two compressions also needs to be estimated. Most existing offset methods are based on traditional approaches, which typically require extracting rich features from the image to build a statistical model. However, when the image size is small, the number of effective features that can be extracted decreases, making it difficult for these algorithms to achieve good performance. Summary of the Invention
[0005] To address the shortcomings mentioned in the background section, the present invention aims to provide a method and system for estimating the offset of unaligned recompressed images based on a dual-branch network.
[0006] The objective of this invention can be achieved through the following technical solution: a method for estimating the offset of unaligned recompressed images based on a dual-branch network, the method comprising the following steps:
[0007] Receive an image, perform fine-grained color expansion on the image to obtain image color channel features, and input the image color channel features into a pre-established second-order difference layer to obtain block effect features;
[0008] The second-order difference layer is input into a pre-established dual-branch network for training to obtain the trained second-order difference layer. The block effect features are then input into the trained second-order difference layer to extract horizontal and vertical features.
[0009] Preferably, the dual-branch network divides the horizontal SOD layer and the vertical SOD layer into two branches of the dual-branch network, and extracts horizontal features and vertical features respectively.
[0010] Preferably, for color images, the Y channel is cropped, and the Cb and Cr channels are also cropped. The calculation formulas for Y, Cb, and Cr are as follows:
[0011]
[0012] Where R, G, and B represent the red, green, and blue channels of a color image, respectively. Although YCbCr represents a color image, the complete color image representation is YCbCrCg. The introduction of the Cg channel information expands the data volume. Cg is calculated from Cb and Cr, and the calculation process is as follows:
[0013] Cg=128-0.417×(Cb-128)-0.865×(Cr-128).
[0014] Preferably, in the process of inputting the image color channel features into a pre-established second-order difference layer to obtain block effect features, the existence of block effect features means that the elements within a block are more correlated, and the elements between blocks are less correlated.
[0015] Preferably, unaligned recompressed images are detected by calculating the LPD of pixels. The LPD is calculated as follows:
[0016] LPD(x,y)=I(x,y)+I(x+1,y+1)-I(x,y+1)-I(x+1,y)
[0017] Where I(x,y) represents the pixel value at position (x,y), I(x+1,y+1) represents the pixel value at position (x+1,y+1), I(x,y+1) represents the pixel value at position (x,y+1), and I(x+1,y) represents the pixel value at position (x+1,y).
[0018] Preferably, the trained second-order difference layer calculates the row and column offsets by calculating the first-order lateral difference H(x,y) and the first-order longitudinal difference V(x,y), respectively:
[0019] H(x,y)=|I(x+1,y)-I(x,y)|
[0020] V(x,y)=|I(x,y+1)-I(x,y)|
[0021] In the formula, H(x,y) is a second-order transverse difference, and V(x,y) is a first-order longitudinal difference.
[0022] Preferably, the horizontal and vertical features obtained by the second-order difference hierarchical extraction after training are as follows:
[0023] The formulas for calculating the second-order horizontal difference H'(x,y) and the second-order vertical difference V'(x,y) are as follows:
[0024] H'(x,y)=|I(x+1,y)-2*I(x,y)+I(x-1,y)|
[0025] V'(x,y)=|I(x,y+1)-2*I(x,y)+I(x,y-1)|
[0026] In the formula, H'(x,y) is the second-order horizontal difference, and V'(x,y) is the second-order vertical difference.
[0027] Secondly, in order to achieve the above objectives, the present invention discloses a system for estimating the offset of unaligned recompressed images based on a dual-branch network, comprising:
[0028] Second-order difference module: used to receive the image, perform fine-grained color expansion on the image to obtain the image color channel features, and input the image color channel features into the pre-established second-order difference layer to obtain the block effect features;
[0029] Dual-branch extraction module: This module is used to input the second-order difference layer into a pre-established dual-branch network for training, thereby obtaining the trained second-order difference layer. Block effect features are then input into the trained second-order difference layer to extract horizontal and vertical features.
[0030] In another aspect of the present invention, in order to achieve the above-mentioned objective, an apparatus is disclosed, comprising:
[0031] One or more processors;
[0032] Memory, used to store one or more programs;
[0033] When one or more of the programs are executed by one or more of the processors, the one or more processors implement the method for estimating the offset of an unaligned recompressed image based on a dual-branch network as described above.
[0034] In another aspect of the invention, in order to achieve the above-mentioned objective, a storage medium containing computer-executable instructions is disclosed, characterized in that the computer-executable instructions, when executed by a computer processor, are used to perform the method described above for estimating the offset of an unaligned recompressed image based on a dual-branch network.
[0035] The beneficial effects of this invention are:
[0036] This invention simulates traditional difference extraction using second-order difference convolution. Convolution is then used during network training to optimize the difference parameters. Furthermore, to address the issue of insufficient feature quantity in small images, the Cg channels of color images are utilized to enrich channel features. Compared to corresponding traditional estimation methods, this method offers higher accuracy and is easier to train. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0038] Figure 1 This is a schematic diagram of the method flow of the present invention;
[0039] Figure 2 This is a schematic diagram of the overall framework of the algorithm of this invention;
[0040] Figure 3 This is a schematic diagram of the optimization process of the second-order difference layering of the present invention;
[0041] Figure 4 This is a schematic diagram of the system structure of the present invention;
[0042] Figure 5 This is a schematic diagram of the non-aligned recompression of the present invention;
[0043] Figure 6 This is an experimental effect verification diagram of the method of the present invention. Detailed Implementation
[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0045] like Figure 1 As shown, a method for estimating the offset of an unaligned recompressed image based on a dual-branch network includes the following steps:
[0046] Receive an image, perform fine-grained color expansion on the image to obtain image color channel features, and input the image color channel features into a pre-established second-order difference layer to obtain block effect features;
[0047] Fine-grained color processing: For color images, if cropping occurs, not only the Y channel is cropped, but the Cb and Cr channels are also cropped similarly. Therefore, color images contain more information than grayscale images. Although YCbCr can represent a color image, the complete representation of a color image is YCbCrCg, and the following formula shows how Y, Cb, and Cr are calculated:
[0048]
[0049] In this context, R, G, and B represent the information of the red, green, and blue channels of a color image, respectively. Although YCbCr represents a color image, the complete color image representation is YCbCrCg, with the introduction of Cg channel information to expand the data volume.
[0050] Since Cg is calculated from Cb and Cr, there is redundancy between the Cg and CbCr channels in terms of numerical statistics. Therefore, previous studies on color images have often neglected the Cg channel. However, in the task of estimating offsets, we are concerned with the cropping marks left by the cropping, rather than statistical data. Since cropping marks exist in the Cb and Cr channels, they also exist in the Cg channel, making the Cg channel a valid source of information reflecting cropping marks. The following equation shows the calculation of Cg using Cb and Cr.
[0051] Cg=128-0.417×(Cb-128)-0.865×(Cr-128)
[0052] Second-order difference layering: In JPEG compression, the image is divided into 8×8 blocks, which introduces block artifacts. The existence of block artifacts means that elements within a block are more correlated, while elements between blocks are less correlated. Based on these characteristics, Chen et al. detected unaligned recompressed images by calculating the pixel-level LPD. The LPD calculation method is as follows:
[0053] LPD(x,y)=I(x,y)+I(x+1,y+1)-I(x,y+1)-I(x+1,y)
[0054] Where I(x,y) represents the pixel value at position (x,y). From the above formula, we can see that LPD considers the differences between rows and columns simultaneously.
[0055] The second-order difference layer is input into a pre-established dual-branch network for training to obtain the trained second-order difference layer. The block effect features are then input into the trained second-order difference layer to extract horizontal and vertical features.
[0056] However, the random cropping that occurs before the second JPEG compression is often random, which results in the offsets between rows and columns being independent. Therefore, when estimating column offsets, interference from row information should be avoided, and when estimating row offsets, interference from column information should be avoided. Hao et al. estimated the row and column offsets by calculating the second-order horizontal difference H(x,y) and the first-order vertical difference V(x,y), respectively.
[0057] H(x,y)=|I(x+1,y)-I(x,y)|
[0058] V(x,y)=|I(x,y+1)-I(x,y)|
[0059] However, first-order differencing only considers the difference between two adjacent pixels, resulting in larger values for first-order differencing in regions with high image texture complexity, which introduces interference when determining grid boundaries. In contrast, second-order differencing overcomes the interference from image texture information. The second-order horizontal differencing H'(x,y) and second-order vertical differencing V'(x,y) are defined as follows:
[0060] H'(x,y)=|I(x+1,y)-2*I(x,y)+I(x-1,y)|
[0061] V'(x,y)=|I(x,y+1)-2*I(x,y)+I(x,y-1)|
[0062] However, the three feature extraction methods mentioned above are all based on experience-based human design. This leads to insufficient mining of cropping trace features, limiting the upper limit of the accuracy of existing algorithms. To overcome this problem, we use convolution to simulate the second-order differencing process and design a second-order differencing layer. This layer can be integrated into the network to participate in overall training, automatically optimizing to find the optimal solution for the differences. As an example, such as... Figure 3 The optimization process of the SOD layer is shown below. First, the four column convolutions are initialized to [1, -2, 1], corresponding to the second-order lateral difference. Then, these four convolutions are used to convolve the red regions of the four channels. Finally, the parameters of the column vectors are continuously optimized during network training. Figure 3 After n optimizations, the network automatically updates the difference parameters.
[0063] The dual-branch network incorporates second-order difference hierarchies into the training of the neural network. The horizontal SOD layer and the vertical SOD layer are divided into two branches of the network, extracting horizontal and vertical features respectively. These two branches differ only in their branch structure at the SOD layer; the subsequent structure of the network is identical. The specific network structure will be explained in detail below.
[0064] Network Structure: Theoretically, deep neural networks can learn complex representations of high-dimensional data, thereby extracting deep information from images. However, deep convolutional networks can learn semantic features of images, which is not suitable for shift estimation tasks. Shallow networks have smaller receptive fields and can capture more details by using finer-grained feature information. Therefore, we chose a shallow network to design the network, as shown in Table 1. One branch of the proposed network consists of horizontal (vertical) convolutional groups, five convolutional layers, a global average pooling layer, a fully connected layer, and a softmax layer. Notably, pooling layers are not used in the network. This is because pooling layers have a large downsampling amplitude, which may lead to the loss of subtle differences in depth features, making shift estimation tasks difficult to learn. However, without downsampling, the network complexity would increase, ultimately making the network difficult to train. To balance the complexity and accuracy of the model, we set the stride of the first convolutional kernel to 2 to achieve downsampling. Furthermore, the downsampling rule is continuously optimized during network training to avoid the network ignoring cropping artifacts in the feature images.
[0065] Table 1. Two-branch network structure
[0066]
[0067] Loss Function: Estimating the misaligned parameters is an 8-class classification problem. The last layer of the network uses a SoftMax layer to map the network output to the [0, 1] interval. ...
[0068]
[0069] Among them, y i p represents the actual unaligned parameter of sample i, and there are 8 categories. i This represents the predicted non-aligned parameter. N represents the number of samples.
[0070] On the other hand, such as Figure 4 As shown, in order to achieve the above objectives, embodiments of the present invention disclose a system for estimating the offset of unaligned recompressed images based on a dual-branch network, comprising:
[0071] Second-order difference module: used to receive the image, perform fine-grained color expansion on the image to obtain the image color channel features, and input the image color channel features into the pre-established second-order difference layer to obtain the block effect features;
[0072] Dual-branch extraction module: This module is used to input the second-order difference layer into a pre-established dual-branch network for training, thereby obtaining the trained second-order difference layer. Block effect features are then input into the trained second-order difference layer to extract horizontal and vertical features.
[0073] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.
[0074] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0075] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0076] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.
Claims
1. A method for estimating the offset of an unaligned recompressed image based on a dual-branch network, characterized in that, Includes the following steps: Acquire the color image to be detected, and convert the R, G, and B channel pixel values of the color image into Y, Cb, and Cr channel pixel values. Based on the Cb and Cr channel pixel values, calculate the Cg channel pixel value using the following formula: Cg = 128 - 0.417×(Cb - 128) - 0.865×(Cr - 128); The pixel values of the four channels Y, Cb, Cr, and Cg are used as the color channel features of the color image; The color channel features are input into a dual-branch network, which includes horizontal and vertical branches. The horizontal branch has a horizontal second-order difference layer at its input end, and the vertical branch has a vertical second-order difference layer at its input end. The parameters of the horizontal second-order difference layer and the vertical second-order difference layer are both trainable parameters. During training, the color channel features of the training sample images are input into the dual-branch network. The horizontal block effect features and vertical block effect features are extracted by the horizontal second-order difference layer and the vertical second-order difference layer, respectively. The horizontal block effect features and the vertical block effect features are then processed by the subsequent layers of the dual-branch network. The parameters of the horizontal second-order difference layer and the vertical second-order difference layer are optimized through training. The color channel features of the image to be detected are input into the trained dual-branch network, which outputs the horizontal and vertical offsets of the image to be detected.
2. A device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When one or more of the programs are executed by one or more of the processors, the one or more of the processors implement the method for estimating the offset of an unaligned recompressed image based on a dual-branch network as described in claim 1.
3. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the method for estimating the offset of an unaligned recompressed image based on a dual-branch network as described in claim 1.