Image tampering detection method and system based on mixed features and RGB features

Abstract

The invention relates to the technical field of digital image security and authentic identification, and provides an image tampering detection method and system based on mixed features and RGB features, and the method comprises the steps: obtaining a to-be-detected input image, and carrying out the preprocessing of the to-be-detected input image; respectively extracting a Haar wavelet high-frequency component, a discrete cosine transform frequency domain feature and a Bayer convolution noise feature, and carrying out matrix level fusion to obtain a mixed feature; extracting RGB (Red, Green and Blue) features for the preprocessed input image; the mixed features are connected through cross-layer residual errors, and mixed feature learning features are obtained; and integrating the mixed feature learning features and the fused RGB features by using a cross-modal feature interaction architecture to obtain a prediction probability graph. Multi-modal features are fused, high-frequency response is enhanced, and the accuracy of image tampering detection is improved by adopting a dynamic fusion mechanism. The technical problems that an existing tampering detection method is insufficient in feature characterization capacity in a complex scene, low in tampering trace detection sensitivity and the like are solved.

Description

Technical Field

[0001] The present invention relates to the technical field related to digital image security and counterfeit identification, and in particular to an image tampering detection method and system based on hybrid features and RGB features. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of artificial intelligence and deep learning technologies, image tampering technology is becoming increasingly automated and intelligent, showing an increasing trend of specialization and covertness. New methods such as deepfakes, partial content replacement, and cross-modal synthesis are becoming increasingly widely used. These methods can achieve high-quality tampering without significantly compromising the overall semantics of the image. The emergence of these technologies has greatly improved the concealment and realism of image tampering, posing severe challenges in a number of key scenarios, including the authenticity verification of news images, the collection of electronic evidence in judicial fields, the anti-counterfeiting of financial instruments, and the regulation of social media content. Therefore, the research and development of accurate and efficient image tampering detection methods has become a critical issue that needs to be addressed in the fields of information security and trusted computing.

[0004] The inventors discovered during their research that while current image tampering detection technology has made some progress, the following issues remain in addressing complex tampering techniques and highly concealed forgeries: 1) The single nature of the feature space leads to the omission of cross-domain tampering evidence: Current mainstream detection methods mostly rely on shallow or single-modal features in the RGB pixel space, such as manual features like Error Level Analysis (ELA) and Color Filter Array (CFA) patterns. While effective in simple scenarios, these methods are less sensitive to high-frequency anomalies (such as edge artifacts and texture breaks) and are susceptible to interference from conventional post-processing operations such as JPEG compression and brightness adjustment, resulting in increased false detection rates. 1) The three-color space (RGB) features are easily disturbed by normal operations such as lighting and color correction, while the use of frequency domain transformations (such as discrete cosine transform and wavelet decomposition) alone faces the problem of a break in the correlation between high-frequency components and spatial semantics; 2) There is an inherent contradiction between network resolution and feature sensitivity: although high-resolution networks can preserve subtle texture differences, they lack a specific response to abnormal noise distribution in tampered areas; deep networks improve classification performance through semantic abstraction, but cause high-frequency tampering traces to be filtered out during the pooling process. For example, the multi-layer downsampling operation in the residual network ResNet will erase fine-grained tampering traces, and the high-resolution network (HRNet) can preserve spatial resolution, but limits sensitivity to details in the tampered area. Existing image tampering detection methods have systematic defects in terms of feature extraction dimension, balance between network resolution and feature sensitivity, resulting in the accuracy and robustness of image tampering detection in complex environments being difficult to meet actual needs. Summary of the Invention

[0005] To address these issues, the present invention proposes an image tampering detection method and system based on hybrid and RGB features. This method integrates multimodal features, enhances high-frequency response, and employs a dynamic fusion mechanism to improve the accuracy of image tampering detection. This method addresses technical issues such as the insufficient feature representation capabilities of existing tampering detection methods in complex scenarios and low sensitivity in detecting tampering traces. The system is applicable to a variety of fields requiring integrity and authenticity verification of digital media, including image authenticity authentication, judicial evidence collection, and news content verification.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] One or more embodiments provide an image tampering detection method based on hybrid features and RGB features, comprising the following steps:

[0008] Get the input image to be detected and perform preprocessing;

[0009] The high-frequency components of Haar wavelet, the frequency domain features of discrete cosine transform and the noise features of Bayer convolution are extracted from the preprocessed input image respectively, and the hybrid features are obtained by matrix-level fusion.

[0010] For the preprocessed input image, parallel multi-scale convolution branches are used to simultaneously extract local detail features and global semantic features to obtain multi-channel RGB features, and then fused based on spatial attention to obtain the fused RGB features;

[0011] The hybrid features are connected across layers to capture multi-scale frequency domain compression traces and noise inconsistency features, and the hybrid feature learning features are obtained;

[0012] The hybrid feature learning features and the fused RGB features adopt a cross-modal feature interaction architecture, and the fusion of multi-scale heterogeneous features is achieved through dynamic weight allocation to obtain a prediction probability map.

[0013] One or more embodiments provide an image tampering detection system based on hybrid features and RGB features, including:

[0014] An image acquisition module is configured to acquire an input image to be detected and perform preprocessing;

[0015] A hybrid feature extraction module is configured to extract the Haar wavelet high-frequency component, discrete cosine transform frequency domain features and Bayer convolution noise features from the preprocessed input image, and perform matrix-level fusion to obtain a hybrid feature;

[0016] The RGB feature extraction module is configured to use parallel multi-scale convolution branches to simultaneously extract local detail features and global semantic features from the preprocessed input image to obtain multi-channel RGB features, and then fuse them based on spatial attention to obtain fused RGB features;

[0017] The hybrid feature learning module is configured to capture multi-scale frequency domain compression traces and noise inconsistency features through cross-layer residual connections of the hybrid features to obtain hybrid feature learning features;

[0018] The RGB branch and hybrid feature branch fusion module is configured to adopt a cross-modal feature interaction architecture to integrate the hybrid feature learning features and the fused RGB features, and realize the fusion of multi-scale heterogeneous features through dynamic weight allocation to obtain a prediction probability map.

[0019] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the above-mentioned image tampering detection method based on mixed features and RGB features are completed.

[0020] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the steps in the above-mentioned image tampering detection method based on mixed features and RGB features are completed.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] The detection method of this embodiment significantly improves the model's ability to detect complex image tampering techniques by introducing multi-source frequency domain features and RGB spatial semantic features. Hybrid feature fusion improves the ability to capture high-frequency anomalies, compression traces, and noise inconsistencies, overcoming the false detection problem caused by the single nature of features in traditional methods. At the same time, the spatial attention mechanism is used to enhance the model's ability to focus on local structures, effectively alleviating the contradiction between high resolution and feature sensitivity. The cross-modal interaction mechanism with dynamic weight regulation further enhances the depth and flexibility of feature fusion, and can maintain high detection accuracy and robustness under a variety of image editing and compression conditions.

[0023] The advantages of the present invention and its additional aspects will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their description are used to explain the present invention but do not constitute a limitation of the present invention.

[0025] Figure 1 Schematic diagram of the overall process of the method for detecting tampered images according to Example 1 of the present invention;

[0026] Figure 2 This is a schematic diagram of the structure of a hybrid feature learning module based on the ResNet-50 network in Example 1 of the present invention;

[0027] Figure 3 This is a schematic diagram of the structure of an RGB feature extraction module based on the HRNet network in Example 1 of the present invention;

[0028] Figure 4 Schematic diagram of the structure of the RGB branch and mixed feature branch fusion module of Example 1 of the present invention; DETAILED DESCRIPTION

[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0030] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0031] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof. It should be noted that, in the absence of conflict, the various embodiments of the present invention and the features in the embodiments can be combined with each other. The embodiments will be described in detail below with reference to the accompanying drawings.

[0032] Example 1

[0033] In the technical solutions disclosed in one or more embodiments, Figures 1 to 4 As shown in FIG, a method for detecting image tampering based on hybrid features and RGB features includes the following steps:

[0034] Step 1: Obtain the input image to be detected and perform preprocessing;

[0035] Step 2: Extract the Haar wavelet high-frequency component, discrete cosine transform frequency domain features and Bayer convolution noise features from the preprocessed input image, and perform matrix-level fusion to obtain hybrid features;

[0036] Step 3: For the preprocessed input image, parallel multi-scale convolution branches are used to synchronously extract local detail features and global semantic features to obtain multi-channel RGB features, and then fused based on spatial attention to obtain the fused RGB features;

[0037] Step 4: The hybrid features are connected across layers to capture multi-scale frequency domain compression traces and noise inconsistency features, and the hybrid feature learning features are obtained;

[0038] Step 5: The hybrid feature learning features and the fused RGB features are combined using a cross-modal feature interaction architecture to achieve the fusion of multi-scale heterogeneous features through dynamic weight allocation to obtain a prediction probability map (mask);

[0039] This method addresses the issues of feature singularity and insufficient feature sensitivity in existing image tampering detection by constructing a detection architecture that fuses multimodal information from the frequency and spatial domains. First, during the preprocessing stage of the input image, the image size and standardized brightness information are unified to establish a consistent foundation for subsequent feature extraction. Haar wavelet high-frequency information, discrete cosine transform features, and Bayer convolution noise maps are extracted based on the image content. These three features reflect image edge changes, compression traces, and photoreceptor noise patterns during the imaging process, respectively. These features are then fused at the matrix level, effectively solving the problem of missing cross-domain tampering evidence in a single feature space.

[0040] At the same time, to enhance the model's sensitivity to the multi-scale structure of images, a parallel multi-scale convolutional branch network was designed to simultaneously extract fine-grained local structure and macro-semantics from images. A spatial attention mechanism was introduced at the feature level to enhance the model's ability to detect features in tampered areas. Subsequently, cross-layer residual connections were used to enhance the hybrid features' ability to detect frequency domain compression traces and noise anomalies at multiple scales, avoiding the loss of high-frequency information caused by deep abstract semantics. Finally, a dynamic weight adjustment strategy was introduced through a cross-modal feature interaction mechanism to effectively fuse feature information of different scales and types under a unified architecture to generate a predicted probability map for tampering localization.

[0041] The detection method of this embodiment significantly improves the model's ability to detect complex image tampering techniques by introducing multi-source frequency domain features and RGB spatial semantic features. Hybrid feature fusion improves the ability to capture high-frequency anomalies, compression traces, and noise inconsistencies, overcoming the false detection problem caused by the single nature of features in traditional methods. At the same time, the spatial attention mechanism is used to enhance the model's ability to focus on local structures, effectively alleviating the contradiction between high resolution and feature sensitivity. The cross-modal interaction mechanism with dynamic weight regulation further enhances the depth and flexibility of feature fusion, and can maintain high detection accuracy and robustness under a variety of image editing and compression conditions.

[0042] Step 1: Get the input image to be detected, perform preprocessing, and standardize the image format:

[0043] Specifically, the format adjustment standard is RGB or grayscale format, and the resolution must meet the set minimum size constraint, such as 512×512 pixels;

[0044] For images from different sources (such as JPEG and PNG formats), the format is first unified and non-RGB images (such as grayscale images) are converted into three-channel RGB format.

[0045] In step 2, the standardized image is processed through three different methods: Haar wavelet decomposition, Bayer convolution, and DCT transform, to obtain three different features, which are then added together to obtain a hybrid feature. The high-frequency components of the Haar wavelet, the frequency domain features of the discrete cosine transform (DCT), and the noise features of the Bayer convolution are innovatively fused at the matrix level. This fusion mechanism can simultaneously capture the characteristics of high-frequency edge distortion (such as splicing artifacts), frequency domain compression traces (such as JPEG blocking effects), and abnormal noise distribution (such as inconsistent local noise patterns) in the tampered area.

[0046] In step 2, the method for generating mixed features includes the following steps:

[0047] Step 21: Perform Haar wavelet decomposition on the pre-processed input image, discard the low-frequency components and perform inverse transformation on the high-frequency components to generate Haar wavelet high-frequency components;

[0048] Step 211: perform three-level Haar wavelet decomposition on the input image to generate four frequency band sub-images (LL, LH, HL, HH) in the horizontal, vertical and diagonal directions;

[0049] The low frequency (LL) and three high frequency sub-bands (LH horizontal, HL vertical, HH diagonal), the decomposition process can be expressed as:

[0050] {LL3, LH3, HL3, HH3} = HWT (I norm ,level=3);

[0051] Where HWT represents the Haar wavelet decomposition function, and level=3 specifies the third-level decomposition depth.

[0052] Step 212: discard the low-frequency component LL sub-image, retain the high-frequency components (LH, HL, HH) containing edge distortion and texture anomalies, and perform inverse wavelet transform to reconstruct the high-frequency enhanced feature map, i.e., Haar wavelet high-frequency component DCT(x);

[0053] In this embodiment, in order to eliminate low-frequency interference in natural scenes (such as smooth backgrounds and uniformly illuminated areas), a low-frequency suppression strategy is adopted: the LL3 subband is discarded, and only high-frequency components (LH3, HL3, and HH3) are retained. Then, an inverse transform is performed, and the formula is as follows:

[0054] F HF=IHWT(LH3,HL3,HH3);

[0055] Among them, IHWT represents inverse Haar wavelet decomposition, F HF It is the high frequency enhancement feature map.

[0056] In this step, the high-frequency enhancement and low-frequency stripping strategies are used to resolve the bottleneck of traditional frequency domain analysis. When directly using the full-band wavelet features, low-frequency components can easily mask high-frequency tampering traces, resulting in loss of detailed information. This embodiment innovatively designs an optimization mechanism: by discarding low-frequency sub-bands through Haar wavelet decomposition, normal low-frequency interference such as lighting changes and natural textures is eliminated. This strategy effectively suppresses the impact of interference factors such as blurring and heavy compression on detection results, significantly improving the positioning accuracy of subtle tampering traces such as edge artifacts and local spectral distortion.

[0057] Step 22, discrete cosine transform (DCT) frequency domain feature extraction: performing discrete cosine transform on the preprocessed input image, converting the input image into the frequency domain to extract global frequency domain features representing global compression traces and spectral anomalies;

[0058] Specifically, a DCT transform is performed on the standardized image to convert the RGB image of size H×W×3 into the frequency domain. After conversion to the frequency domain, a high-pass filter is used for filtering, and the cutoff frequency of the high-pass filter is set to 20% of the Nyquist frequency. Then, an inverse transform (IDCT) is performed to restore it to the spatial domain, and the global frequency domain abnormal features of the processed image are extracted.

[0059] Step 23, Bayer convolution noise feature extraction: performing noise feature extraction on the preprocessed input image based on the deformable convolution layer of the Bayer filter array to obtain the Bayer convolution noise feature Bayer(x);

[0060] Specifically, a 3×3 deformable convolution kernel is constructed to extract the noise features of the H×W×3 RGB image;

[0061] In this step, the deformable convolutional layer adaptively matches the image sensor noise pattern, and the noise distribution difference characteristics unique to the tampered area can be extracted.

[0062] In step 2, the three different feature maps are added and fused at the matrix level to obtain hybrid features, which can achieve the coordinated enhancement of 3D evidence of high-frequency artifacts, frequency domain anomalies and noise distortion. The formula is as follows:

[0063] F mix =DCT(x)+Bayer(x)+F HF ;

[0064] Traditional methods usually analyze frequency domain or spatial domain features in isolation, making it difficult to fully capture complex anomalies in tampered areas. This embodiment actively removes low-frequency interference through Haar wavelet decomposition, retains high-frequency edge distortion features in the horizontal / vertical directions, and integrates them with the DCT global frequency domain features across scales. At the same time, an improved Bayer convolution layer is introduced to enhance noise sensitivity, enabling multi-dimensional correlation analysis of high-frequency artifacts, frequency domain compression traces, and noise distribution anomalies. Compared to single-modality detection methods, this embodiment can simultaneously identify complex tampering types such as deep fakes, local erasures, and cross-scale splicing, significantly improving the detection recall rate while maintaining a low false detection rate.

[0065] Steps 3 and 4 are implemented by constructing a dual-branch network, which includes two parallel branches. The first branch is the ResNet network, referred to as the hybrid branch; the second branch is the HRNet network, referred to as the RGB branch. The first branch learns the detail information in the hybrid features at high resolution, and the latter extracts multi-scale spatial details, and finally realizes tampering positioning through the cross-modal fusion module.

[0066] In step 3, the second branch forms an RGB feature extraction module based on the HRNet network to extract RGB features.

[0067] The RGB feature extraction module uses parallel multi-scale convolution branches (HRNet network) to simultaneously extract local detail features (such as edge breaks and texture mutations) and global semantic features (such as lighting consistency and color distortion), and then fuses them based on spatial attention to obtain fused RGB features.

[0068] Specifically, the second branch receives a preprocessed, standardized input image as input. After learning, its neural network outputs four spatial features at different scales: F1, F2, F3, and F4. This branch utilizes the HRNet network architecture as its backbone, notably maintaining parallel processing of multi-resolution features and enabling efficient interaction of feature streams through cross-scale connection modules.

[0069] In some embodiments, the RGB feature extraction module includes multiple scale convolution branches set from high resolution to low resolution, each convolution branch includes one or more convolution layers, and a pooling layer at the output end. Different convolution branches are interactively fused through branch increments and cross-scale interactions at different processing stages; the output ends of other convolution branches except the highest resolution convolution branch are connected to the channel control module and the SE attention module, and the fusion is based on the introduction of a top-down spatial attention mechanism, and the RGB features are obtained after upsampling.

[0070] Specifically, such as Figure 3 As shown in Figure 2, the image is fed into convolution branches of multiple scales in the HRNet network. Figure 3Taking the four convolutional branches as an example, from top to bottom they are the first, second, third, and fourth convolutional branches. The low-resolution path is responsible for capturing semantic hierarchical information, while the high-resolution path is used to preserve edge and spatial details. Features at each scale are repeatedly transferred and reorganized through feature fusion modules, resulting in a final fused feature map with both rich semantics and precise positioning capabilities. This is particularly critical for detecting small, blurred, tampered areas in images.

[0071] At the network architecture level, the RGB feature branch constructs a hierarchical multi-resolution processing system, employing a progressive branch expansion strategy to achieve a dynamic balance between detail preservation and semantic deepening. This branch comprises four feature processing stages, forming a comprehensive processing flow from high-resolution fine-grained features to low-resolution semantic features. Each stage constructs a structure with progressively increasing resolution through branch increments and cross-scale interactions.

[0072] As a specific example, the RGB feature extraction module of this embodiment includes four stages, which are described as follows:

[0073] In the first processing stage, the basic feature flow is initialized as Figure 3 The first column in the image is a standard image input with a size of 64×64×64. The image features are initially extracted through the convolution layer, and a feature map with a resolution of 32×32 is generated. The number of channels is expanded from 64 to 128, which is passed to the second stage.

[0074] Specifically, the design limits the receptive field of the initial convolutional layer (3×3) to avoid premature semantic abstraction that leads to loss of high-frequency details, and retains complete spatial topology information for subsequent tampering trace analysis.

[0075] The second processing stage is Figure 3 In the second column of Figure 1, a new resolution branch is added in this stage. The first convolution branch continues processing, and a second convolution branch is added with a size of 32×32×128 (the spatial resolution is half of branch 1, and the number of channels is twice that of branch 1). From this stage, multi-scale branches are introduced to enhance the network's perception of objects of different scales through gradual downsampling and channel expansion.

[0076] Starting from the second stage, the network gradually introduces new resolution branches to build multi-scale perception capabilities. When entering the nth processing stage (n ≥ 2), the system adds a new resolution branch: the spatial dimension of this branch is reduced to 1 / 2 of the previous highest resolution branch, and the number of channels is expanded to twice that of the previous branch.

[0077] In the third processing stage, a new third convolutional branch with a size of 16×16×256 is added. It continues the principle of "halving the resolution and doubling the channels" to guide the network to acquire more abstract and global low-resolution features while running in parallel with the high-resolution branch. Keeping the three branches running in parallel, they continue to deepen the processing of features at different scales.

[0078] In the fourth processing stage, a new fourth convolution branch is added with a size of 8×8×512;

[0079] In view of the fact that features of different scales extracted by HRNet output features of different scales, the obtained feature maps of four different scales are input into the top-down spatial attention multi-scale feature fusion method to obtain the fused RGB features: a top-down spatial attention mechanism is introduced in the RGB branch, and the 1 / 4, 1 / 8, 1 / 16, and 1 / 32 scale feature maps output by HRNet are fed into the improved SE module, so that the richer spatial information in the high-resolution branch can guide other branches.

[0080] The RGB branch outputs four different RGB features as input, and the HRNet branch fusion module can fuse the features of four different scales to obtain a unified RGB feature. After the RGB image passes through the HRNet network, the four branches output features of different sizes and numbers of channels. In order to make full use of the spatial detail information contained in the high-resolution branch and enhance the low-resolution branch's ability to pay attention to the boundaries and positions of the tampered areas, this embodiment proposes a top-down spatial attention fusion strategy. Before feature splicing, the fusion module introduces a spatial attention mechanism to guide the high-semantic branch to learn to pay attention to the spatial structure information provided by the low-semantic branch;

[0081] In step 3, multi-channel RGB features are obtained and fused based on spatial attention to obtain fused RGB features, which includes the following steps:

[0082] Step 31: downsample the feature map with a higher resolution so that the size of the feature map with a higher resolution is the same as that of the feature map with a lower resolution.

[0083] For example, the RGB feature map output by the first convolution branch is 64×64×64, and the RGB features of the first convolution branch are downsampled so that the RGB feature map of the first branch is 32×32×64;

[0084] Step 32: Adjust the channels of the feature map of the lower resolution so that the number of channels of the feature map of the lower resolution is the same as the number of channels of the feature map of the higher resolution;

[0085] The RGB feature map output by the first convolution branch is 64×64×64, with 64 channels. The RGB feature map of the second branch (32×32×128) is channel-adjusted so that the number of channels of the second branch is 64.

[0086] Step 33: Use the attention mechanism (SE module) to generate a spatial weight map for the downsampled high-resolution feature map;

[0087] Step 34: performing weighted fusion based on the obtained weight map and the downsampled feature map with a lower resolution to obtain an updated feature map with a higher resolution;

[0088] Step 35: After upsampling all the feature maps with higher resolution updated after weighted fusion, they are spliced ​​and fused to obtain the fused RGB features;

[0089] In the above steps, the higher-level resolution feature map and the lower-level resolution feature map refer to the feature maps output by adjacent branches.

[0090] The following is an example of a specific four-stage branch to explain the specific process of steps 31 to 35;

[0091] First, the first convolution branch (64×64×64) is used as the branch with the richest spatial information, and spatial attention is enhanced on the second convolution branch (32×32×128). To this end, all feature channels are first adjusted to the size of the first convolution branch through convolution. The first convolution branch is downsampled once to make its size consistent with the second convolution branch (32×32). Then, the SE module is used to generate a spatial weight map, and the feature map of the second branch is weighted. This allows it to retain semantic information while introducing detail perception capabilities in high-resolution features. The weighted operation formula is as follows:

[0092] F′1=Downsample(F1);

[0093] F′2=sigmoid(Conv(concat(max_pool(F′1),avg_pool(F′1))))·F2;

[0094] Next, the enhanced second convolution branch is applied to the third convolution branch in the same manner, that is, its feature map is downsampled to 16×16, and the third convolution branch is guided to learn the spatial weight distribution, thereby enhancing the spatial structure response of low- and medium-resolution features. Similarly, the third convolution branch continues to apply spatial attention guidance to the fourth branch to complete the final level of feature enhancement. Through this top-down spatial attention mechanism, low-resolution, high-semantic deep features are able to retain more edge details and position information, enhancing their ability to respond to tiny tampered areas.

[0095] Finally, the four feature maps adjusted by the spatial attention mechanism are upsampled to a unified spatial size (64×64) and concatenated in the channel dimension to form a unified fused feature map, namely the fused RGB feature. This fused feature map simultaneously contains multi-scale and multi-level semantic information and spatial details, providing a strong representational foundation for the subsequent complementary fusion with the mixed features of the first convolutional branch, resulting in a unified RGB feature.

[0096] In step 4, the second branch of the dual-branch network forms a hybrid feature learning module based on the ResNet-50 network, which fuses the high-frequency components of the Haar wavelet, the frequency domain features of the discrete cosine transform, and the noise features of the Bayer convolution. The hybrid features are connected through residual connections and deformable convolutions to capture multi-scale frequency domain compression traces and noise inconsistency characteristics;

[0097] To further enhance the model's ability to detect minor tampering in images, this branch of the present embodiment introduces a lightweight, high-resolution feature extraction network. This network can employ a high-receptive-field convolutional network based on an improved ResNet50 architecture, including four cascaded residual layers.

[0098] After hybrid feature extraction, such as Figure 2 As shown in Figure 1, the input image, after being standardized in the hybrid branch input format, undergoes Haar wavelet decomposition, DCT frequency domain analysis, and Bayer convolution in step 2 to obtain a hybrid feature that combines high-frequency enhancement information, frequency domain features, and noise characteristics. This hybrid feature is input into a ResNet-based neural network (consisting of four stages) to obtain the feature map extracted by the hybrid feature branch.

[0099] The network maintains the resolution of feature maps and uses multi-scale convolutional layers and residual connections, including multiple residual blocks. Each residual block performs a deformable convolution after multiple residual connection operations, effectively capturing local texture details and large-scale contextual semantics.

[0100] In this embodiment, the ResNet-50 network used includes four residual layers. The number of residual blocks from layer 1 to layer 4 is 3, 4, 6, and 3 respectively. The structure of each residual block performs a residual operation on the input features, which includes convolution, layer normalization, and activation operations in sequence to obtain an updated feature map. The residual operation formula is:

[0101] X=relu(bn(conv(relu(bn(conv(X))))))+X);

[0102] Among them, X is the input feature map;

[0103] In addition, to solve the problem of complex background interference in the image, a deformable convolution is inserted after each stage layer. The deformable convolution kernel is used to build long-range spatial dependencies, thereby improving the response strength of the tampered area.

[0104] Deformable convolution is expressed as:

[0105] Y deform =∑ m,n X(i+Δx m,n ,j+Δy m,n )·W(m,n);

[0106] Among them, X represents the input feature map, W(m,n) represents the weight of the convolution kernel, m,n represent the relative coordinates within the convolution kernel, Δy m,n and Δx m,n Represents the spatial offset of dynamic learning, and each sampling position of the convolution kernel is (i+Δx m,n , j+Δy m,n ); i, j represents the position coordinates of the output feature map; Y deform Represents the feature map output after the deformable convolution operation;

[0107] Deformable convolution, when the convolution kernel and the feature map are convolved and sliding element-by-element multiplied, the window is no longer a rectangle like the convolution kernel, and the corresponding multiplication and addition are no longer performed in position, but can be offset.

[0108] For example, if three residual blocks are set in layer 1, the feature map in layer 1 undergoes three residual operations and then a deformable convolution. If four residual blocks are set in layer 2, the feature map in layer 2 undergoes four residual operations and then a deformable convolution. The other layers are processed in the same way. Layers 2 to 4 are not sampled, so the image size remains unchanged.

[0109] After the mixed feature passes through layer 1 to layer 4, the mixed feature map learned by the mixed feature extraction branch is obtained.

[0110] This embodiment builds a dual-branch spatial-frequency domain network architecture. In the spatial domain, a multi-scale HRNet network is used, preserving local details and global semantic features at the original resolution through parallel convolution branches. In the frequency domain, wavelet high-frequency features are fused with DCT frequency-domain features to form a highly discriminative hybrid representation. The multi-scale spatial attention module dynamically integrates multi-scale features, avoiding computational redundancy caused by channel splicing while precisely enhancing key features.

[0111] Step 5 is achieved by constructing a fusion module of RGB branch and mixed feature branch, as shown in Figure 4As shown in the figure, the hybrid feature learning features and the fused RGB features are taken as input, and the final tampering detection probability map is obtained through the RGB branch and the hybrid feature branch fusion module;

[0112] The method of using a cross-modal feature interaction architecture to integrate hybrid feature learning features and fused RGB features, and implementing the fusion of multi-scale heterogeneous features through dynamic weight allocation to obtain a prediction probability map includes the following steps:

[0113] Step 51: The fused RGB feature map and the mixed feature learning feature map are combined through channel cascade operation, and the cross attention mechanism is used to realize dynamic feature weighting;

[0114] Step 52: Perform channel concatenation on the features after the cross-attention operation, perform channel compression and nonlinear activation through convolution, and then output the predicted probability map after the activation operation.

[0115] Optionally, the RGB branch and the mixed feature branch fusion module include a cross-attention module and a feature fusion and dimensionality reduction module; the cross-attention module includes a forward attention module and a reverse attention module; the deep integration of multi-scale heterogeneous features is achieved through dynamic weight allocation of the forward attention module and the reverse attention module, and the modal difference problem between the spatial domain detail features and the frequency domain noise features is solved through feature alignment.

[0116] The cross attention module is configured to dynamically weight features by using the cross attention mechanism on the aligned RGB feature map (H×W×256) and the mixed feature learning feature map (H×W×512) through channel cascade operation.

[0117] Specifically, the frequency domain features of the hybrid feature learning feature map output by the hybrid branch are used as the query vector (Query), and the spatial domain features output by the RGB branch are used as the key-value pair (Key-Value). The spatial attention weight is generated by calculating the cross-modal similarity matrix.

[0118] The cross-attention mechanism enables the network to automatically focus on the spatial details corresponding to the frequency domain abnormal areas. For example, the forward attention module strengthens the response strength of edge discontinuity features at locations with significant DCT blocking effects. At the same time, the reverse attention module uses RGB features as query vectors to mine noise patterns related to local texture abnormalities in the mixed features, forming a bidirectional feature enhancement effect. The formula of the cross-attention mechanism is:

[0119] The forward attention formula is:

[0120]

[0121] The reverse attention formula is:

[0122]

[0123] Among them, F freq Represents hybrid learning features, and constructs the feature map output by the hybrid feature learning module based on the ResNet-50 network;

[0124] Q freq represents the query vector extracted based on hybrid learning features, obtained by weight matrix projection;

[0125] represents the key vector extracted based on hybrid learning features, which is obtained by projecting and then transposing the weight matrix;

[0126] F rgb Represents the fused RGB features output by the RGB feature extraction module based on the HRNet network;

[0127] Q rgb Represents the query vector extracted based on the fused RGB features, which is obtained by projecting the weight matrix.

[0128] Represents the key vector extracted based on the fused RGB features, which is obtained by projecting the weight matrix and then transposing it.

[0129] represents the scaling factor, which is obtained by dividing The dot product result is scaled to prevent the dot product value from being too large, which causes the gradient of Softmax to disappear.

[0130] The feature fusion and dimensionality reduction module is configured to perform channel concatenation on the generated features, perform channel compression and nonlinear activation through 1×1 convolution, and finally activate the output through bilinear interpolation and sigmoid function to output the predicted probability map;

[0131] The image tampering detection model that implements steps 1 to 5 above includes an image acquisition module, a hybrid feature extraction module, a dual-branch network, and an RGB branch and hybrid feature branch fusion module;

[0132] A dual-branch network, including an RGB feature extraction module based on the HRNet network and a hybrid feature learning module based on the ResNet-50 network;

[0133] An image acquisition module is configured to acquire an input image to be detected and perform preprocessing;

[0134] A hybrid feature extraction module is configured to extract the Haar wavelet high-frequency component, discrete cosine transform frequency domain features and Bayer convolution noise features from the preprocessed input image, and perform matrix-level fusion to obtain a hybrid feature;

[0135] The RGB feature extraction module is configured to use parallel multi-scale convolution branches to extract local detail features and global semantic features from the pre-processed input image to obtain multi-channel RGB features, and then fuse them based on spatial attention to obtain the fused RGB features; the network structure is as follows Figure 3 As shown;

[0136] The hybrid feature learning module is configured to capture multi-scale frequency domain compression traces and noise inconsistency features through cross-layer residual connections to obtain hybrid feature learning features; the network structure is as follows Figure 2 As shown;

[0137] The RGB branch and hybrid feature branch fusion module is configured to use a cross-modal feature interaction architecture to integrate the hybrid feature learning features and the fused RGB features, and achieve the fusion of multi-scale heterogeneous features through dynamic weight allocation to obtain a prediction probability map; the network structure is as follows Figure 4 As shown;

[0138] During the training process, the input image undergoes format standardization, mixed feature processing, a dual-branch network, and a feature fusion module to obtain the output prediction probability map. The final output is trained using a loss function consisting of cross entropy and dice loss, as shown in the following formula:

[0139] Loss = 0.3 × loss dice +0.7×loss bce ;

[0140] The network is trained using the loss function and the Adam optimizer.

[0141] The tampered image detection method of this embodiment first standardizes and normalizes the tampered image, and extracts noise characteristics through Bayer convolution. The original image is used to remove low-frequency information to emphasize high-frequency information, and feature extraction is performed through a high-resolution branch network to avoid the loss of detail information. The discrete cosine transform (DCT) is used to extract frequency domain information. For RGB information, it contains not only detail features but also global features, so a multi-scale feature extraction network is used. This method can make full use of forensic information of different characteristics in the image and improve the accuracy of detection. After passing through the dual-branch network, the learned features are fused, and the predicted tampering probability map is finally output.

[0142] Example 2

[0143] Based on Example 1, this embodiment provides an image tampering detection system based on hybrid features and RGB features, including:

[0144] An image acquisition module is configured to acquire an input image to be detected and perform preprocessing;

[0145] A hybrid feature extraction module is configured to extract the Haar wavelet high-frequency component, discrete cosine transform frequency domain features and Bayer convolution noise features from the preprocessed input image, and perform matrix-level fusion to obtain a hybrid feature;

[0146] The RGB feature extraction module is configured to use parallel multi-scale convolution branches to simultaneously extract local detail features and global semantic features from the preprocessed input image to obtain multi-channel RGB features, and then fuse them based on spatial attention to obtain fused RGB features;

[0147] The hybrid feature learning module is configured to capture multi-scale frequency domain compression traces and noise inconsistency features through cross-layer residual connections of the hybrid features to obtain hybrid feature learning features;

[0148] The RGB branch and hybrid feature branch fusion module is configured to adopt a cross-modal feature interaction architecture to integrate the hybrid feature learning features and the fused RGB features, and realize the fusion of multi-scale heterogeneous features through dynamic weight allocation to obtain a prediction probability map.

[0149] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation processes are the same, which will not be repeated here.

[0150] Example 3

[0151] This embodiment provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps of the image tampering detection method based on hybrid features and RGB features in Example 1 are completed.

[0152] Example 4

[0153] This embodiment provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps of the image tampering detection method based on mixed features and RGB features in Example 1 are completed.

[0154] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

[0155] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. The image tampering detection method based on hybrid features and RGB features is characterized by: The steps include: Get the input image to be detected and perform preprocessing; The high-frequency components of Haar wavelet, the frequency domain features of discrete cosine transform and the noise features of Bayer convolution are extracted from the preprocessed input image respectively, and the hybrid features are obtained by matrix-level fusion. For the preprocessed input image, parallel multi-scale convolution branches are used to simultaneously extract local detail features and global semantic features to obtain multi-channel RGB features, and then fused based on spatial attention to obtain the fused RGB features; The hybrid features are connected across layers to capture multi-scale frequency domain compression traces and noise inconsistency features, and the hybrid feature learning features are obtained; The hybrid feature learning features and the fused RGB features adopt a cross-modal feature interaction architecture, and the fusion of multi-scale heterogeneous features is achieved through dynamic weight allocation to obtain a prediction probability map.

2. The image tampering detection method based on hybrid features and RGB features according to claim 1, characterized in that: The method for generating mixed features includes the following steps: Perform Haar wavelet decomposition on the preprocessed input image, discard the low-frequency components and perform adaptive weighted inverse transformation on the high-frequency components to generate Haar wavelet high-frequency components; Perform discrete cosine transform on the preprocessed input image, convert the input image into the frequency domain, and extract the global frequency domain features that represent global compression traces and spectral anomalies, namely discrete cosine transform frequency domain features; The preprocessed input image is subjected to noise feature extraction based on the deformable convolution layer of the Bayer filter array to obtain the Bayer convolution noise feature; The high-frequency components of Haar wavelet, discrete cosine transform frequency domain features and Bayer convolution noise features are added and fused at matrix level to obtain hybrid features.

3. The image tampering detection method based on hybrid features and RGB features according to claim 2, characterized in that: The method for generating the high frequency component of Haar wavelet includes: Perform three-level Haar wavelet decomposition on the input image to generate four frequency band sub-images in horizontal, vertical and diagonal directions; The low-frequency component sub-image is discarded, and the high-frequency component containing edge distortion and texture anomaly is retained. The inverse wavelet transform is performed to reconstruct the high-frequency enhanced feature map, namely the Haar wavelet high-frequency component.

4. The image tampering detection method based on hybrid features and RGB features according to claim 1, wherein: Build an RGB feature extraction module based on the HRNet network to extract RGB features; The RGB feature extraction module includes multiple scale convolution branches set from high resolution to low resolution. Each convolution branch includes one or more convolution layers and a pooling layer at the output end. Different convolution branches are interactively fused through branch increments and cross-scale interactions at different processing stages. The output ends of other convolution branches except the highest resolution convolution branch are connected to the channel control module and the SE attention module. The fusion is based on the introduction of a top-down spatial attention mechanism, and the RGB features are obtained after upsampling.

5. The image tampering detection method based on hybrid features and RGB features according to claim 1, characterized in that: The multi-channel RGB features are obtained and fused based on spatial attention to obtain the fused RGB features, including the following steps: Downsample the feature map of the higher resolution so that the size of the feature map of the higher resolution is the same as that of the feature map of the lower resolution; Adjust the number of channels of the feature map of the lower resolution so that the number of channels of the lower resolution is the same as that of the feature map of the higher resolution; The downsampled higher-resolution feature map is used to generate a spatial weight map using the attention mechanism; The feature map of the higher resolution is weightedly fused with the downsampled feature map of the lower resolution based on the obtained weight map to obtain an updated feature map of the higher resolution; All the feature maps with higher resolution updated after weighted fusion are upsampled and then concatenated to obtain the fused RGB features.

6. The image tampering detection method based on hybrid features and RGB features according to claim 1, wherein: A hybrid feature learning module is constructed based on the ResNet-50 network. The hybrid features obtained by fusing the high-frequency components of the Haar wavelet, the frequency domain features of the discrete cosine transform and the Bayer convolution noise features are captured through cross-layer residual connections to obtain the hybrid feature learning features.

7. The image tampering detection method based on hybrid features and RGB features according to claim 1, characterized in that: The method of using a cross-modal feature interaction architecture to integrate hybrid feature learning features and fused RGB features, and implementing the fusion of multi-scale heterogeneous features through dynamic weight allocation to obtain a prediction probability map includes the following steps: The fused RGB feature map and the hybrid feature learning feature map are combined through channel cascade operation, and the cross attention mechanism is used to achieve dynamic feature weighting; The features after the cross-attention operation are channel-concatenated, channel compression and nonlinear activation are performed through convolution, and then after the activation operation, the predicted probability map is output.

8. Image tampering detection system based on hybrid features and RGB features, characterized by: include: An image acquisition module is configured to acquire an input image to be detected and perform preprocessing; A hybrid feature extraction module is configured to extract the Haar wavelet high-frequency component, discrete cosine transform frequency domain features and Bayer convolution noise features from the preprocessed input image, and perform matrix-level fusion to obtain a hybrid feature; The RGB feature extraction module is configured to use parallel multi-scale convolution branches to simultaneously extract local detail features and global semantic features from the preprocessed input image to obtain multi-channel RGB features, and then fuse them based on spatial attention to obtain fused RGB features; The hybrid feature learning module is configured to capture multi-scale frequency domain compression traces and noise inconsistency features through cross-layer residual connections of the hybrid features to obtain hybrid feature learning features; The RGB branch and hybrid feature branch fusion module is configured to adopt a cross-modal feature interaction architecture to integrate the hybrid feature learning features and the fused RGB features, and realize the fusion of multi-scale heterogeneous features through dynamic weight allocation to obtain a prediction probability map.

9. An electronic device, characterized in that: The invention comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the steps of the image tampering detection method based on hybrid features and RGB features according to any one of claims 1 to 7 are completed.

10. A computer-readable storage medium, characterized in that Used to store computer instructions, which, when executed by a processor, complete the steps of the image tampering detection method based on mixed features and RGB features as described in any one of claims 1 to 7.

Images