An Image Forgery Classification Method Based on Multi-Level Attribute Nodes

Abstract

The present invention relates to an image forgery classification method based on multi-order attribute nodes. From the comprehensive perspective of image processing, the edge attributes, texture attributes, grayscale attributes, and color attributes of an image are extracted in different ways. At the same time, considering the change in the amount of information before and after image processing and the characteristics of the attributes themselves, the first-order entropy processing is performed on the edge attributes of the image, the second-order entropy processing is performed on the texture attributes, and the local entropy processing is performed on the grayscale and color attributes. Finally, the entropies of the four attributes are represented as a hierarchical and progressive graph structure relationship, and a graph convolutional network is introduced as the classification backbone network to classify all manipulated images and unmanipulated images.

Description

Technical Field

[0001] The present invention relates to an image forgery classification method, and more particularly to an image forgery classification method based on multi-order attribute nodes. Background Art

[0002] In the past few years, in order to cope with image forgery, the academic community has proposed many methods for identifying the authenticity of images, such as image forensics, deep learning, and adversarial learning methods. However, most of these methods perform network training by analyzing a single attribute of the image. Their recognition ability usually has certain limitations. They can only analyze specific image attributes and cannot perform comprehensive analysis. When operating on image media, a local operation will be performed instead of a global operation, which will surely damage the overall structure of the image. For example, the edge gradient is unbalanced, the texture structure is damaged, the gray-scale statistics are abnormal, and the color matching is distorted. Although there are several image post-processing methods, these damaged structures cannot be completely repaired.

[0003] For a specific image attribute, its performance in a real image and a manipulated image should be different. For example, in the operation of a face image, there may be a relatively obvious stitching boundary between the face and the background, and the texture of the face part may be different from normal; the overall color of the image may also be unbalanced. For these possible situations, human vision can directly judge the authenticity under normal circumstances. However, with the upgrade of post-processing technology, these phenomena can no longer be captured by human vision, so machine vision is needed. However, due to the continuous update of artificial intelligence technology, machine vision for a single attribute gradually fails to obtain convincing evidence. Therefore, developing an image forgery classification method for multiple image attributes has become an urgent problem to be solved. Summary of the Invention

[0004] The object of the present invention is to propose an image forgery classification method based on multi-order attribute nodes, and the technical solution is as follows:

[0005] The first step: perform image attribute separation

[0006] Any image can be divided into four attributes, namely edge attribute, texture attribute, gray-scale attribute, and color attribute. The present invention uses different methods to separate the above four attributes.

[0007] (1) Use the Laplacian of a Gaussian (LoG) to separate the edge attribute of the image. First, apply a Gaussian filter (G×f) to the image, then perform the Laplacian operator operation Δ(G×f), retain the position of the peak of its first derivative, find the Laplacian zero-crossing point from it, and finally perform interpolation and estimate the accurate position of the zero-crossing point.

[0008] (2) The texture attributes of the image are separated using the coarseness, contrast, and directionality in Tamura texture features.

[0009] (3) Gamma transformation is used to enhance the details of the dark part, and gray-level change statistics are performed on phenomena such as overexposure or underexposure caused by image manipulation, thereby separating the gray-level attributes of the image.

[0010] (4) The image is converted to the YCbCr color space to evaluate its quality change, and the color attributes of the image are separated.

[0011] Through the separation of the above four attributes, the overall impact of image manipulation is decomposed into the impacts of different attributes. Through analysis, we can obtain the manipulation trajectories under each attribute, and then make a comprehensive judgment on the information changes of genuine and fake images.

[0012] The second step: Calculate the information entropy of the obtained image attributes.

[0013] Calculate the first-order entropy of the image for the edge attributes, calculate the second-order entropy of the image for the texture attributes, and calculate the local image entropy for the gray-level and color attributes.

[0014] By calculating the multi-order information entropy of the four image attributes, information nodes representing different attributes can be obtained, and the information entropy of all manipulated images and unmanipulated images is calculated to form a set of information nodes for different attributes.

[0015] The third step: Establish a hierarchical progressive graph structure

[0016] In order to enhance the connection between different attribute nodes, according to the characteristics of the first-order entropy, second-order entropy, and local entropy, a hierarchical graph structure information relationship based on the first-order entropy is proposed, where the second-order entropy is used as the axis and the two local entropies are the scattered points.

[0017] The graph convolutional network GCN is introduced as the backbone network for the information node topic classification task. Since the hierarchical progressive graph structure layout has been performed on the image attribute information nodes, the number of layers of the introduced graph convolutional network only needs to be set to 15 layers. Finally, all manipulated images and unmanipulated images are fed into the graph convolutional network according to the graph structure relationship for training classification. Description of the Drawings

[0018] Figure 1 It is the overall architecture of the method in the present invention, including attribute separation, multi-order information entropy calculation, and graph structure relationship. Detailed Implementation Manner

[0019] To make the objectives and technical solutions of the present invention clearer, the specific implementation steps of the present invention will be described in detail below with reference to the accompanying drawings. The specific implementation manners are as follows:

[0020] Step 1: Perform image attribute separation

[0021] Any image can be divided into four attributes, namely edge attribute, texture attribute, grayscale attribute, and color attribute. The present invention separates each attribute using different methods.

[0022] (1) Use the Laplacian of a Gaussian (LoG) to separate the edge attribute of the image. First, apply a Gaussian filter (G×f) to the image, then perform the Laplacian operator operation Δ(G×f), retain the positions of the first derivative peaks, find the Laplacian zero-crossings from them, and finally perform interpolation and estimate the accurate positions of the zero-crossings. The specific calculation process is as follows:

[0023] (1.1) Apply a Gaussian filter (G×f) to the image:

[0024]

[0025] (1.2) Perform the Laplacian operator operation Δ(G×f) and retain the positions of the first derivative peaks:

[0026] LOG(f)(x, y) = Δ(G σ *f) = ΔG σ *f (2)

[0027] (1.3) Perform interpolation and estimate the accurate positions of the zero-crossings:

[0028]

[0029] where σ is a scale parameter. The larger σ is, the more blurred the image is, and the better the noise filtering effect is. The smaller σ is, the opposite is true. Thus, we obtain the edge attribute of the image.

[0030] (2) Use the coarseness, contrast, and directionality in Tamura texture features to separate the texture attribute of the image. The specific calculation process is as follows:

[0031] (2.1) Coarseness is a quantity that reflects the grain size in the texture and is the most basic texture feature. When the knowledge primitive sizes of two texture feature patterns are different, the pattern with the larger primitive size gives a coarser feeling. Its calculation can be divided into the following steps:

[0032] (2.1.1) Calculate the average intensity value of the pixels in a sliding window of size 2k×2k pixels in the image:

[0033]

[0034] (2.1.2) For each pixel, calculate the average intensity difference between non-overlapping horizontal and vertical windows:

[0035] E k，h (m, n) = Ak(m + 2 {k-1}，n)-A_{k}(m-2 {k - 1}, n) (5)

[0036] E k，v (m, n) = Ak(m, n + 2 {k-1})-A_{k}(m，n-2 {k - 1}) (6)

[0037] Where, for each pixel, the k value that maximizes the E value is used to set the optimal size S best (m, n) = 2k.

[0038] (2.1.3) By calculating the average S of the entire image best , the roughness of the image can be obtained:

[0039]

[0040] (2.2) Contrast is calculated by computing the distribution of pixel intensities and provides a global measure of the entire image. The calculation formula is as follows:

[0041]

[0042] Where μ4 is the fourth-order moment and σ 2 is the variance.

[0043] (2.3) Directionality is a global property of a given texture region that describes how the texture property diverges or converges along certain directions. Its calculation can be divided into the following steps:

[0044] (2.3.1) First, calculate the gradient vector of each pixel:

[0045]

[0046]

[0047] Where all pixel gradient vectors θ are represented by the histogram H. The histogram is used to discretize the range of θ and calculate the number of pixels whose value is greater than a given threshold |ΔG|.

[0048] (2.3.2) By calculating the sharpness of the histogram peak, the image orientation can be obtained, and the calculation formula is as follows:

[0049]

[0050] Where p represents the peak in the histogram, and n p is all the peaks in the histogram. For a certain peak p, W p represents all the discrete regions included in this peak, and Φ p is the position of the center of the wave peak.

[0051] (3) Use γ transformation to enhance the details of the dark part, and perform gray-scale change statistics on phenomena such as overexposure or underexposure caused by image manipulation, so as to separate the gray-scale attributes of the image. The calculation formula is as follows:

[0052] Dt = c * (D + ∈) γ (12)

[0053] When 0 < γ < 1, the low-gray-scale area of the image is enlarged, the high-gray-scale area is reduced, and the contrast of the image is increased; when γ > 1, the area with higher gray scale in the image is enlarged, the part with lower gray scale is reduced, and the contrast of the image is reduced.

[0054] (4) The image itself is in the RGB color model. When the image is manipulated, the manipulated part will still return to the RGB space, which is not sensitive to color changes. Therefore, it is necessary to convert the image to the YCbCr color space to evaluate its quality change. After converting from the RGB to the YCbCr color space, the color attributes of the image can be separated, and machine vision can more easily capture the imperceptible color changes brought by image manipulation.

[0055]

[0056] Where Y represents the luminance component, Cb represents the chrominance component of blue, and Cr represents the chrominance component of red.

[0057] Through the separation of the above four attributes, the overall impact of image manipulation is decomposed into the impacts of different attributes. Through analysis, we can obtain the manipulation trajectory under each attribute, and then make a comprehensive judgment on the information changes of real and fake images.

[0058] The second step: Calculate the information entropy of the obtained image attributes.

[0059] Calculate the first-order entropy of the image for the edge attribute, calculate the second-order entropy of the image for the texture attribute, and calculate the local image entropy for the gray-scale and color attributes. The specific calculation process is as follows:

[0060] (1) For edge attributes, the information changes are concentrated in different directions, so the first-order entropy of the image is calculated for them.

[0061] Let \(p_a\) represent the proportion of pixels with gray value \(a\) in the image. Then the first-order entropy of the image is defined as:

[0062]

[0063] where \(p_a\) is the probability that each gray level appears in the image.

[0064] (2) For texture attributes, the information changes are mainly reflected in the global pixel changes. Therefore, the second-order entropy of the image is calculated for them. The neighborhood gray values of the image are selected as the spatial feature quantities of the gray distribution. Different pixel gray values form multiple feature pairs, denoted as \((x, y)\), where \(x\) represents the gray value of the pixel (\(0\leq x\leq255\)) and \(y\) represents the average value of the neighborhood gray levels (\(0\leq y\leq255\)):

[0065]

[0066] This formula reflects the linkage characteristics between the gray value at a certain pixel position and the surrounding distribution, where \(f(x, y)\) is the feature frequency and \(N\) is the image scale.

[0067] The second-order entropy of the discrete image is:

[0068]

[0069] (3) For gray-scale and color attributes, the information changes are mainly reflected in the block aggregation areas. Therefore, the local image entropy is calculated for them.

[0070]

[0071] where \(f(x, y)\) represents the gray level at point \((x, y)\) in the image, and \(p\) xy is the distribution probability of the gray level at point \((x, y)\). \(U\times V\) is regarded as the local neighborhood centered at \((x, y)\) in the image, and \(H\) is regarded as the local entropy value of the image.

[0072] By calculating the multi-order information entropy of the four image attributes, information nodes representing different attributes can be obtained, and the information entropy of all manipulated images and unmanipulated images is calculated to form two different sets of attribute information nodes.

[0073] Step 3: Establish a hierarchical progressive graph structure

[0074] To enhance the connection between different attribute nodes, according to the characteristics of the first-order entropy, second-order entropy, and local entropy, a hierarchical graph structure information relationship based on the first-order entropy is proposed, where the second-order entropy is used as the axis and the two local entropies are the scattered points. SeeFigure 1 。

[0075] (1) To utilize the established hierarchical progressive information directed graph structure relationship, a traditional graph convolutional network is introduced, which is expressed by the following formula:

[0076] O layer+1 = f(O layer , A) (18)

[0077] o0 = X is the first layer (X ∈ R N*D ), N is the number of graph structure connection points, D is the dimensionality of the feature vector of each node, and A is the adjacency matrix containing convolutional relationships.

[0078] (2) Let f be the Laplacian matrix. Change the above formula to:

[0079]

[0080] This can solve two problems. One is to introduce a self-measurement matrix to solve the self-feedback problem; the other is to normalize the adjacency matrix, where the bilateral adjacency matrix is multiplied by the square degree of the nodes and then inverted.

[0081] (3) For each pair of nodes, the elements in the matrix are obtained as follows:

[0082]

[0083] where deg(vi) and deg(vj) are the node degrees

[0084] Introduce the graph convolutional network GCN as the backbone network for the information node topic classification task. Since the hierarchical progressive graph structure layout has been performed on the image attribute information nodes, the number of layers of the introduced graph convolutional network only needs to be set to 15 layers. Finally, all the manipulated images and unmanipulated images are fed into the graph convolutional network according to the graph structure relationship for training classification.

Claims

1. An image forgery classification method based on multi - order attribute nodes, characterized in that: The first step: Perform image attribute separation The image is divided into four attributes, namely edge attribute, texture attribute, grayscale attribute and color attribute, and different methods are used to separate the above four attributes; (1) Use the Laplacian of a Gaussian (LoG) to separate the edge attribute of the image. First, apply the Gaussian filter G×f to the image, then perform the Laplacian operator operation △(G×f), retain the positions of the first - order derivative peaks, and find the Laplacian zero - crossings from them. Finally, perform interpolation and estimate the exact positions of the zero - crossings; (2) Use the coarseness, contrast and directionality in Tamura texture features to separate the texture attribute of the image; (3) Use γ - transformation to enhance the details of the dark part and perform grayscale change statistics on phenomena such as over - exposure or under - exposure caused by image manipulation, so as to separate the grayscale attribute of the image; (4) Convert the image to the YCbCr color space to evaluate its quality change and separate the color attribute of the image; Through the separation of the above four attributes, the overall impact of image manipulation is decomposed into the impacts of different attributes. Through analysis, the manipulation trajectories under each attribute are obtained, and then a comprehensive judgment is made on the information changes between real and fake images; The second step: Calculate the information entropy of the obtained image attributes Calculate the first - order entropy of the image for the edge attribute, calculate the second - order entropy of the image for the texture attribute, and calculate the local image entropy for the grayscale attribute and color attribute; By calculating the multi - order information entropy of the four image attributes, information nodes representing different attributes can be obtained, and the information entropy of all manipulated images and unmanipulated images is calculated to form a set of information nodes of different attributes; The third step: Establish a hierarchical progressive graph structure According to the characteristics of the first - order entropy, second - order entropy and local entropy, establish a hierarchical graph structure information relationship based on the first - order entropy, where the second - order entropy is used as the axis and the two local entropies are scattered points; Introduce the graph convolutional network GCN as the backbone network for the information node topic classification task. The number of layers of the graph convolutional network is set to 15 layers. Finally, all manipulated images and unmanipulated images are fed into the graph convolutional network according to the graph structure relationship for training classification.

Images