Unknown forged image detection method based on stack generalization ensemble learning, medium and terminal

Abstract

The invention relates to an unknown forged image detection method based on stack generalization ensemble learning, a medium and a terminal, and in the primary learning stage of stack generalization ensemble learning, a data set is input into two classification networks for a traditional forged image and a deep forged image. A Convnextv2-based multi-source feature enhancement model is provided for a traditional forged image to carry out forging detection; a space and channel reconstruction convolution and Transform hybrid network model is proposed for a deep counterfeit image to carry out counterfeit detection. The two classification networks are trained through a k-fold cross validation method to obtain a plurality of primary learners, and prediction results of the primary learners are input into a meta-learner fractional order Kolmogorov-Arnold network in a stacking generalization integrated learning meta-learning stage to be trained. According to the method, the defect that single model prediction is easy to fit to single distribution is overcome, and the detection precision for unknown forged images in a real environment is improved.

Description

Technical Field

[0001] The present invention relates to the fields of computer image processing and artificial intelligence, and in particular to an unknown forged image detection method, medium and terminal based on stacked generalized ensemble learning. Background Art

[0002] With the development of digital image processing technology, it is becoming easier to create fake images, and the means of forgery are becoming more and more diverse. Traditional image forensics methods mainly rely on analyzing specific features of images, such as noise patterns, compression artifacts, etc., but these methods are often unable to cope with fake images generated by deep learning. Deep fake technologies, such as generative adversarial networks (GANs) and diffusion models, can generate images with extremely high fidelity, making the detection of fake images more difficult.

[0003] The current forged image detection technology has the following main shortcomings:

[0004] (1) Single model limitations: Most existing detection methods rely on a single model or algorithm, which may perform well on specific types of forged images but may fail for unknown or new forgery techniques.

[0005] (2) Insufficient generalization ability: Existing technologies are often optimized for specific counterfeiting methods and lack sufficient generalization ability, making it difficult to adapt to the ever-changing counterfeiting techniques.

[0006] (3) Limited detection accuracy: Due to the continuous advancement of counterfeiting technology, the accuracy of traditional detection methods is often unsatisfactory when identifying deep fake images.

[0007] (4) Lack of multimodal fusion: Many detection methods fail to make full use of multimodal data, such as combining the RGB information and noise characteristics of the image to improve the accuracy and robustness of detection.

[0008] In order to meet these challenges, this paper proposes a method for detecting unknown forged images based on stacked generalization ensemble learning. This method integrates multiple traditional forged image detection models with deep forged image detection models, and uses a stacked generalization strategy to integrate multiple models. It uses an adaptive differential evolution algorithm to measure and optimize the weights of different models, effectively combining them to improve the detection accuracy of unknown forged images, enhance the generalization ability of the model, and reduce the adverse effects of forged images on individuals and society. Summary of the invention

[0009] In view of the shortcomings of the prior art, the present invention proposes an unknown forged image detection method based on stacked generalized ensemble learning.

[0010] An object of the present invention is to provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes an unknown forged image detection method based on stacked generalized ensemble learning.

[0011] Another object of the present invention is to provide an information data processing terminal, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the unknown forged image detection method based on stacked generalized ensemble learning.

[0012] A method for detecting unknown forged images based on stacked generalized ensemble learning, comprising:

[0013] The first learning stage is primary learning, which is to input the data set into two classification network models for traditional fake images and deep fake images, including a multi-source feature enhancement model based on Convnextv2 for traditional fake images and a spatial and channel reconstruction convolution and Transformer hybrid network model for deep fake images. The two classification networks are trained by k-fold cross-validation method to obtain multiple primary learners; output prediction results

[0014] The second learning stage is meta-learning, which is to input the prediction results of the primary learner into the meta-learner fractional-order Kolmogorov-Arnold network to train and obtain the final integrated model;

[0015] Input the real-time image to the final integrated model and output the predicted unknown forged image result.

[0016] Preferably, the data set includes a training set and a test set, and both the training set and the test set include RGB image sources;

[0017] The specific purpose of obtaining the data set is to collect and organize the publicly available tampered image sample sets and self-made sample sets using different forgery methods, including publicly available traditional and deep forgery image sample sets and self-made sample sets using traditional forgery methods and deep forgery methods. The training set is input into a multi-source feature enhancement model based on Convnextv2, and the test set is input into a spatial and channel reconstruction convolution and Transformer hybrid network model.

[0018] As a preferred method, a multi-source feature enhancement model based on Convnextv2 is used for forgery detection. Independent weights are trained to extract features. The network architecture of each image source uses Convnextv2 as the feature extraction backbone network, and a residual self-supervised prediction convolution attention module is added to enhance feature representation.

[0019] The ordinary convolution and spatial and channel reconstruction convolution modules are combined as feature extractors to obtain salient features from the input image, and the feature maps are divided into blocks and assembled into block embedding sequences and sent to the Transformer module with deep convolutional multi-layer perceptron for feature extraction and classification;

[0020] Preferably, the training set and test set images are input as primary training set and test set, and the training set is divided using k-fold cross validation, so as to train multiple primary learners through the detection model for traditional forgery methods or deep forgery methods:

[0021] Preferably, the collecting and collating of the disclosed tampered image sample sets using different forgery methods comprises:

[0022] Traditional forgeries include CASIAv2, NIST16, IMD2020, DEFACTO public traditional forgery datasets, and self-made traditional forgery datasets generated by background segmentation and edge smoothing tampering methods and produced by Photoshop software;

[0023] Deep fakes include the CocoGlide public fake dataset and locally tampered datasets generated with the Stable Diffusion and Gen-2 deep generative models.

[0024] Preferably, multiple primary learners are trained by the detection model for traditional forgery methods, and forgery detection is performed based on the multi-source feature enhancement model of Convnextv2 for traditional forgery methods, wherein:

[0025] For the three image sources of RGB image source, SRM filtered noise image source and Bayer convolution filtered noise image source, the same network architecture is used to train independent weights to extract features;

[0026] The network architecture of each image source uses Convnextv2 as the feature extraction backbone network, and adds a residual self-supervised prediction convolution attention module to enhance feature representation and predict or reconstruct tampered information. The decoder uses a simple 3×3 convolution kernel, and the classification head uses the mapping of the decoder after dropout.

[0027] A multi-source consistency learning strategy is adopted to weighted average the prediction results of different image sources to generate fused predictions as pseudo-true labels to guide model training of other image sources.

[0028] Preferably, multiple primary learners are trained through a detection model for deep fake methods, and deep fake methods are used to perform fake detection based on a spatial and channel reconstruction convolution and Transformer hybrid network model, wherein:

[0029] The common convolution is combined with the spatial and channel reconstruction convolution modules as a feature extractor to obtain salient features from the input image. Specifically, 19 common convolutions and 1 spatial and channel reconstruction convolution module are used in the front part of the model to reduce redundant calculations and promote representative feature learning.

[0030] The extracted image salient feature maps are divided into blocks, assembled into block embedding sequences and sent to the improved Transformer module for feature extraction. The improved Transformer module uses a multi-layer perceptron with deep convolution and multi-head self-attention blocks, and replaces the fixed-size position encoding with zero-filled position encoding;

[0031] The output layer consists of 2 channels, including the two classes of real and fake, and the output of the MLP head is processed using the Softmax function to produce probability values ​​in the range of 0 to 1.

[0032] Preferably, a meta-learner is trained by integrating multiple primary learners trained by k-fold cross-validation for traditional forgery method or deep forgery method detection model training, and the training of the meta-learner includes:

[0033] Concatenate the validation set output of the primary learner with the test set output to obtain a set of predicted labels and predicted probabilities, and add the true labels to this set as the input of the meta-learner training set;

[0034] A fractional-order Kolmogorov-Arnold network model is proposed as a meta-learner, and the back-propagation algorithm is used for parameter tuning. The tuned meta-learner is used for prediction to obtain the prediction results.

[0035] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the above-mentioned unknown forged image detection method based on stacked generalized ensemble learning.

[0036] An information data processing terminal comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the above-mentioned unknown forged image detection method based on stacked generalized ensemble learning.

[0037] Combining all the above technical solutions, the advantages and positive effects of the present invention are as follows:

[0038] (1) The present invention uses multiple data sets to train primary learners with multiple network architectures for detecting forged images. Different primary learners have different focuses, and different network architectures are used for images generated by different forgery methods. Then, a stacked generalization ensemble learning method that can combine different models and integrate learning is introduced, and an adaptive differential evolution algorithm is used to search for the optimal weight distribution of different models, and different models are fused, thereby making up for the shortcomings of single model prediction and effectively improving the accuracy and generalization of prediction.

[0039] (2) The multi-source feature enhancement model based on Convnextv2 for traditional forged images in the present invention enhances the model's ability to predict and reconstruct tampered information in images by combining predictions from different information sources and adding a residual self-supervised prediction convolutional attention module. This can improve the model's generalization ability for images tampered with by unknown forgery techniques and improve detection robustness.

[0040] (3) The spatial and channel reconstruction convolution and Transformer hybrid network model for deep fake images in the present invention combines ordinary convolution with spatial and channel reconstruction convolution modules as feature extractors to obtain significant features from input images, reduce redundant calculations, and use a Transformer module with a deep convolutional multi-layer perceptron for feature extraction and classification, thereby improving the detection accuracy of deep fake images. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 This is an overall architecture diagram of unknown forged image detection based on stacked generalized ensemble learning according to an embodiment of the present invention.

[0043] Figure 2 This is a fractional-order Jacobi neural block architecture diagram of an embodiment of the present invention.

[0044] Figure 3 The present invention is a flow chart of a k-fold cross-validation training primary learner.

[0045] Figure 4 This is an architecture diagram of a multi-source feature enhancement model based on Convnextv2 in an embodiment of the present invention.

[0046] Figure 5 This is a residual self-supervised prediction convolutional attention block architecture diagram of an embodiment of the present invention.

[0047] Figure 6 This is a spatial and channel reconstruction convolution and Transformer hybrid network model architecture diagram of an embodiment of the present invention.

[0048] Figure 7 This is a standard convolutional block network architecture diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] In view of the problems existing in the prior art, the present invention provides an unknown forged image detection method based on stacked generalized ensemble learning.

[0051] 1. First, the principle of the method of the present invention is introduced.

[0052] The present invention is implemented as follows: a method for detecting unknown forged images based on stacked generalized ensemble learning, comprising:

[0053] Step 1: Collect and organize the publicly available sample sets of tampered images using different forgery methods and self-made sample sets, including the publicly available sample sets of traditional and deep forgery images and self-made sample sets using traditional forgery methods and deep forgery methods, to form training sets and test sets; Collect and organize the publicly available sample sets of tampered images using different forgery methods, including:

[0054] In terms of traditional forgery, we introduced public traditional forgery datasets such as CASIAv2, NIST16, IMD2020, DEFACTO, as well as self-made traditional forgery datasets generated by background segmentation and edge smoothing tampering methods and produced by Photoshop software; in terms of deep forgery, we introduced the CocoGlide public forgery dataset and the local tampering dataset generated by Stable Diffusion and Gen-2 deep generation models.

[0055] Step 2: In the primary learning stage of ensemble learning, input the training set and test set images as the primary training set and test set, use k-fold cross validation to divide the training set, and then train multiple primary learners through the detection model for traditional forgery methods or deep forgery methods; the classification network model used as the primary learner includes the following two models:

[0056] (1) Aiming at the traditional forgery methods, a multi-source feature enhancement model based on Convnextv2 is proposed for forgery detection. The multi-source feature enhancement model based on Convnextv2 mainly includes the following features:

[0057] 1.1) For the three image sources of RGB image source, SRM filtered noise image source and Bayer convolution filtered noise image source, use the same network architecture and train independent weights to extract features;

[0058] 1.2) The RGB image source provides rich color and texture information, which is very useful for detecting abnormal areas in the image (areas that may have been manipulated); the SRM filtered noise image source can capture the noise distribution in the image that may have changed due to tampering operations, and assist in detecting tampered areas; the Bayer convolution filtered noise image source can reveal the changes in the image compared to natural noise characteristics by learning the noise pattern in the image, and assist in detecting whether the image has been edited unnaturally.

[0059] 1.3) The network architecture of each image source uses Convnextv2 as the feature extraction backbone network, and adds a residual self-supervised prediction convolution attention module to enhance feature representation and predict or reconstruct tampered information. The decoder uses a simple 3×3 convolution kernel, and the classification head uses the mapping of the decoder after dropout (temporary retreat method);

[0060] 1.4) A multi-source consistency learning strategy is adopted to weighted average the prediction results of different image sources to generate fused predictions as pseudo-true labels to guide model training of other image sources.

[0061] (2) A spatial and channel reconstruction convolution and Transformer hybrid network model is proposed for deep fake detection. The spatial and channel reconstruction convolution and Transformer hybrid network model mainly includes the following features:

[0062] 2.1) Use ordinary convolutions in combination with spatial and channel reconstruction convolution modules as feature extractors to obtain salient features from the input image. Specifically, 19 ordinary convolutions and 1 spatial and channel reconstruction convolution module are used in the front part of the model to reduce redundant calculations and promote representative feature learning;

[0063] 2.2) The image salient feature maps extracted by the above operation are divided into blocks, assembled into block embedding sequences and sent to the improved Transformer module for feature extraction. The improved Transformer module uses a multi-layer perceptron with deep convolution and a multi-head self-attention block, and replaces the fixed-size position encoding with zero-filled position encoding;

[0064] 2.3) The output layer consists of 2 channels, including the two classes of real and fake. The output of the MLP head is processed using the Softmax function to produce a probability value in the range of 0 to 1.

[0065] Step 3: In the meta-learning stage of ensemble learning, multiple primary learners trained for traditional forgery methods or deep forgery detection models are integrated, and the outputs of the primary learners are combined to train the meta-learner, which uses a fractional-order Kolmogorov-Arnold network to construct the final ensemble model. Training the meta-learner includes the following steps:

[0066] 1) Concatenate the validation set output of the primary learner with the test set output to obtain a set of predicted labels and predicted probabilities, and add the true labels to this set as the input of the meta-learner training set;

[0067] 2) A fractional-order Kolmogorov-Arnold network model is proposed as a meta-learner, and the back-propagation algorithm is used for parameter tuning. The tuned meta-learner is used for prediction to obtain the prediction results.

[0068] Step 4: Input the real-time image into the final integrated model and output the predicted unknown forged image result.

[0069] Second, the present invention is described in detail below with reference to the accompanying drawings.

[0070] like Figure 1 As shown in FIG2 , the unknown forged image detection method based on stacked generalized ensemble learning of the present invention first collects and organizes the tampered image sample sets and self-made sample sets of different public forgery methods to form training sets and test sets; then inputs the images as primary training sets and test sets, and trains multiple classification models as primary learners in the ensemble learning stage; finally, multiple primary learners are integrated, the meta-learner weights are trained, and the weight distribution of different primary learners output is optimized by using an adaptive differential evolution algorithm to construct a final ensemble model.

[0071] The specific steps can be divided into the following steps:

[0072] (1) Preparation of primary data set: In order to implement the unknown forged image detection method based on stacked generalized ensemble learning proposed in this invention, it is first necessary to collect and organize the public tampered image sample sets and self-made sample sets of different forgery methods. Public data sets include traditional forged image data sets such as CASIAv2, NIST16, IMD2020, DEFACTO, CocoGlide, and deep forged image data sets made by deep generation methods such as Stable Diffusion or ProGAN. The self-made sample set simulates different forgery methods by reprocessing the public image data set COCO2017 through traditional methods such as splicing, copying and moving, and deep generation models such as diffusion and GAN. The constructed data set contains low-resolution and high-resolution image samples to meet the detection needs of different scales.

[0073] (2) Primary data set division: The primary data set is divided into a training set and a test set, and the primary training set T is randomly divided into k subsets T1,…,T k , where a subset T m (m=1,…,k) are used as validation sets, and the remaining subsets are used as training sets. This can form a combination as a primary sub-training set so that k-fold cross-validation training can be used in the primary learner training process.

[0074] (3) Primary learner training: Use k primary sub-training sets to input the traditional fake image classification network and the deep fake image classification network for training, and use the test set for testing. A total of 2k primary learners and 2k validation set test results V on these primary learners can be obtained. 11 ,...,V 1k and V 21 ,...,V 2k And the test results R of 10 test sets 11 ,...,R 1k and R 21 ,...,R 2k .

[0075] (4) Metadata set preparation: V in step (3) 11 ,...,V 1k Connect in series to get V1 = (V 11 ,...,V 15 ) T , V 21 ,...,V 2k Connect in series to get V2 = (V 21 ,...,V 25 ) T , test set test results R 11 ,...,R 15 After taking the average value, we get R 21 ,...,R 25 After taking the average, we get R2. The training set V in the meta-learning phase is ((V1, V2), Y), and the test set R is ((R1, R2), Y'), where Y is the true label of the initial training data and Y' is the true label of the initial test data.

[0076] (5) Training and prediction of meta-learner: Based on the meta-training set V and meta-test set R obtained in step (4), a fractional-order Kolmogorov-Arnold network model is proposed as a meta-learner. The back-propagation algorithm is used for parameter tuning. The tuned meta-learner is used for prediction to obtain the prediction results.

[0077] The Kolmogorov-Arnold network structure is similar to the multilayer perceptron, except that the multilayer perceptron has a fixed activation function on the nodes (neurons), while the Kolmogorov-Arnold network has a learnable activation function on the edges (weights), thereby improving the interpretability and classification accuracy of the network. Based on the Kolmogorov-Arnold network architecture, the present invention uses a trainable adaptive fractional-order Jacobian function as an activation function to further improve the classification performance of the network. The training process of the fractional-order Kolmogorov-Arnold network is as follows:

[0078] a. Initialize all parameters in the network, including α, β, and γ in the fractional Jacobi neural block, as well as the weights and biases of other layers. The fractional Jacobi function is used as the activation function in the fractional Jacobi neural block. The trainable parameters are α, β, and fractional γ. The fractional Jacobi function is expressed as: The parameters α and β are adjusted by the ELU function to ensure convergence, and the fractional order γ is adjusted by the Sigmoid function to ensure its positive value.

[0079] b. Use the back propagation algorithm to calculate the gradient of each parameter, thereby adjusting the parameters and reducing the loss;

[0080] c. Update the parameters using the L-BFGS algorithm based on the gradient.

[0081] By leveraging the unique mathematical properties of fractional Jacobian functions, including simple derivative formulas, non-polynomial behavior, and liveness for both positive and negative input values, efficient learning and high accuracy of the meta-learner are ensured.

[0082] like Figure 3 As shown, in the unknown forged image detection method based on stacked generalized ensemble learning of the present invention, the detection model for traditional forged images is a multi-source feature enhanced forged detection model based on Convnextv2, and its main training process is: for each image source, train a set of independent weights to extract features, use Convnextv2 as the feature extraction backbone network, and add residual self-supervised prediction convolution attention blocks to enhance feature representation and predict or reconstruct tampered information; then generate a fused prediction by weighted averaging the prediction results of different models; use the fused prediction as a pseudo-true label to guide the training of the model of each individual image source.

[0083] The specific steps can be divided into the following steps:

[0084] (1) Data source preparation: Select different image sources, including original RGB images, noise maps generated using SRM (Steganalysis Rich Model) filters, and noise maps generated using Bayer convolution filters. RGB images are the most basic image representation, containing information from the three color channels of the image: red, green, and blue. They provide rich color and texture information, which is very useful for detecting abnormal areas (areas that may have been manipulated) in images. SRM is a noise model used for image steganalysis. It uses a predefined kernel to learn different types of noise residuals of pixels around the central pixel, thereby capturing the noise distribution in the image that may have changed due to tampering operations. For example, copy-move operations may introduce unnatural noise patterns in the image. SRM noise maps can highlight these anomalies and help detect tampered areas. Bayer noise maps are image noise features based on the Bayer pattern. The Bayer pattern is an arrangement of image sensors whose noise characteristics are different from the natural noise characteristics of the image. Since different image editing or synthesis techniques may have different effects on the sensor noise pattern, the Bayer noise map can reveal these changes by learning the noise patterns in the image and assist in detecting whether the image has undergone unnatural editing.

[0085] (2) Independent model training: For each data source, an independent model is built. These models do not share parameters during training, but share the same training scheme. The model uses Convnextv2 as an encoder to extract high-level feature representations of the image; after the encoder, a residual self-supervised prediction convolution attention block is introduced to enhance feature representation and predict or reconstruct tampered information; the decoder uses a simple 3×3 convolution kernel, and the classification head is a mapping of the decoder after dropout.

[0086] (3) Residual self-supervised prediction convolutional attention block (such as Figure 4 As shown in the figure): The residual self-supervised prediction convolutional attention block is mainly composed of a masked convolution layer and a channel attention module, and a residual connection is established before the masked convolution operation and after the channel attention operation to promote the information flow and gradient propagation within the module. CNN frameworks such as Convnextv2 can extract hierarchical features from low-level features (edges, corners) to high-level features (semantic features), but lack the ability to understand and integrate these local features. The residual self-supervised prediction convolutional attention block can learn to use contextual information to predict or reconstruct masked (tampered) information.

[0087] (4) Fusion prediction generation: integrated prediction The fusion prediction is generated by weighted averaging the prediction results of each independent model. The integrated prediction calculation strategy is as follows:

[0088]

[0089] The weights here are predefined and are used to balance the predictions of different models.

[0090] (5) Pseudo-real mask generation: The fusion prediction is binarized into a pseudo-real mask by setting a threshold θ (pseudo ground truth), thereby providing pixel-level supervision signals for each independent model. The pseudo ground truth mask is calculated as follows:

[0091]

[0092] is the pixel-level response in the fused prediction map.

[0093] (6) Back-propagation and model optimization: Use the generated pseudo-real masks to guide the binary cross entropy loss of each individual model Such as the formula:

[0094]

[0095] (H×W is the size of the prediction graph, is the predicted response of the model at position (i, j) from different sources), plus the binary cross entropy loss between the true label and the adaptive pooling prediction To update the parameters of each independent model ( is the image-level prediction determined by adaptive pooling to minimize the prediction error.

[0096] Through the above steps, the predictions of different information sources can be effectively combined, the generalization ability of the model for unknown counterfeiting techniques can be improved, and the detection robustness of the model can be improved.

[0097] like Figure 5 As shown, in the unknown forged image detection method based on stacked generalized ensemble learning of the present invention, a Transformer-based spatial and channel reconstruction convolution hybrid model is proposed for deep forged images, and its main training process is:

[0098] (1) Ordinary convolution and spatial and channel reconstruction convolution (SCConv) modules are used in combination as feature extractors to extract salient features from the input image. To improve efficiency and reduce model parameters and floating point operations (FLOPs), we use SCConv and standard convolution stacks for effective feature extraction. We use 19 standard convolutions and 1 SCConv in the front of the model, with the SCConv being placed after the first standard convolution layer. The convolution module stack is shown in the figure. To prevent inter-layer modifications that may hinder the learning dynamics of the CNN architecture, the stride of the convolution kernel is 1 and the padding is 1. After each standard convolution, batch normalization and Mish activation function processing are performed, which can significantly improve the generalization and optimization capabilities of the model. In addition, the above processing is also followed by a maximum pooling layer with a window size of 2×2 and a stride of 2, which effectively reduces the computational complexity.

[0099] (2) The image feature map processed by SCConv is divided into blocks, and then assembled into a block embedding sequence and sent to the improved Transformer multi-head self-attention layer for feature extraction. Six Transformer encoders are stacked to increase the nonlinear expression ability of the model, helping the model to better learn and represent deeper features and distinguish subtle differences between true and false.

[0100] (3) The improved Transformer encoder uses a deep convolutional multi-layer perceptron (DW-MLP) and a multi-head self-attention block. The main features are that the fixed-size positional encoding is replaced by zero-filled positional encoding, allowing the model to process inputs of different sizes; and a DW convolution with a padding size of 1 is added between the first fully connected (FC) layer and the GELU activation layer of the feedforward network. Due to its deep separation characteristics, the DW convolution can reduce the amount of model calculation and the number of parameters, improve computational efficiency, and provide a certain degree of regularization effect, which helps to reduce overfitting and improve the generalization ability of the model.

[0101] (4) The output layer consists of 2 channels, representing the two classes of real and fake. The Softmax function operates on the output of the MLP head and produces a probability value in the range of 0 to 1. A score close to 1 is used as an indicator of enhanced model performance. The logarithmic loss function is used, and the formula is as follows:

[0102]

[0103] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A method for detecting unknown forged images based on stacked generalized ensemble learning, characterized in that: include: The first learning stage is primary learning, which is to input the data set into two classification network models for traditional fake images and deep fake images, including a multi-source feature enhancement model based on Convnextv2 for traditional fake images and a spatial and channel reconstruction convolution and Transformer hybrid network model for deep fake images. The two classification networks are trained by k-fold cross-validation method to obtain multiple primary learners; output prediction results The second learning stage is meta-learning, which is to input the prediction results of the primary learner into the meta-learner fractional-order Kolmogorov-Arnold network to train and obtain the final integrated model; Input the real-time image to the final integrated model and output the predicted unknown forged image result.

2. The unknown forged image detection method based on stacked generalized ensemble learning as claimed in claim 1, characterized in that: The dataset includes a training set and a test set, both of which include RGB image sources; The specific purpose of obtaining the data set is to collect and organize the publicly available tampered image sample sets and self-made sample sets using different forgery methods, including publicly available traditional and deep forgery image sample sets and self-made sample sets using traditional forgery methods and deep forgery methods. The training set is input into a multi-source feature enhancement model based on Convnextv2, and the test set is input into a spatial and channel reconstruction convolution and Transformer hybrid network model.

3. The unknown forged image detection method based on stacked generalized ensemble learning as claimed in claim 1, characterized in that: The multi-source feature enhancement model based on Convnextv2 is used for forgery detection. Independent weights are trained to extract features. The network architecture of each image source uses Convnextv2 as the feature extraction backbone network, and adds a residual self-supervised prediction convolution attention module to enhance feature representation. The ordinary convolution and the spatial and channel reconstruction convolution modules are combined as feature extractors to obtain salient features from the input image, and the feature maps are divided into blocks and assembled into block embedding sequences and sent to the Transformer module with deep convolutional multi-layer perceptron for feature extraction and classification.

4. The unknown forged image detection method based on stacked generalized ensemble learning as claimed in claim 1, characterized in that: The training set and test set images are input as the primary training set and test set, and the training set is divided using k-fold cross validation, so as to train multiple primary learners through the detection model for traditional counterfeiting methods or deep counterfeiting methods.

5. The unknown forged image detection method based on stacked generalized ensemble learning as claimed in claim 1, characterized in that: The collected and organized public tampered image sample sets using different forgery methods include: Traditional forgeries include CASIAv2, NIST16, IMD2020, DEFACTO public traditional forgery datasets, and self-made traditional forgery datasets generated by background segmentation and edge smoothing tampering methods and produced by Photoshop software; Deep fakes include the CocoGlide public fake dataset and locally tampered datasets generated with the Stable Diffusion and Gen-2 deep generative models.

6. The unknown forged image detection method based on stacked generalized ensemble learning as claimed in claim 1, characterized in that: After training multiple primary learners based on the detection model for traditional forgery methods, forgery detection is performed based on the multi-source feature enhancement model of Convnextv2 for traditional forgery methods. For the three image sources of RGB image source, SRM filtered noise image source and Bayer convolution filtered noise image source, the same network architecture is used to train independent weights to extract features; The network architecture of each image source uses Convnextv2 as the feature extraction backbone network, and adds a residual self-supervised prediction convolution attention module to enhance feature representation and predict or reconstruct tampered information. The decoder uses a simple 3×3 convolution kernel, and the classification head uses the mapping of the decoder after dropout. A multi-source consistency learning strategy is adopted to weighted average the prediction results of different image sources to generate fused predictions as pseudo-true labels to guide model training of other image sources.

7. The unknown forged image detection method based on stacked generalized ensemble learning as claimed in claim 1, characterized in that: After training multiple primary learners through the detection model for deep fake methods, deep fake methods are used to detect fakes based on spatial and channel reconstruction convolution and Transformer hybrid network models, among which, The common convolution is combined with the spatial and channel reconstruction convolution modules as a feature extractor to obtain salient features from the input image. Specifically, 19 common convolutions and 1 spatial and channel reconstruction convolution module are used in the front part of the model to reduce redundant calculations and promote representative feature learning. The extracted image salient feature maps are divided into blocks, assembled into block embedding sequences and sent to the improved Transformer module for feature extraction. The improved Transformer module uses a multi-layer perceptron with deep convolution and multi-head self-attention blocks, and replaces the fixed-size position encoding with zero-filled position encoding; The output layer consists of 2 channels, including the two classes of real and fake, and the output of the MLP head is processed using the Softmax function to produce probability values ​​in the range of 0 to 1.

8. The unknown forged image detection method based on stacked generalized ensemble learning as claimed in claim 1, characterized in that: A meta-learner is trained by integrating multiple primary learners trained by k-fold cross-validation for traditional forgery methods or deep forgery method detection model training. The training meta-learner includes: Concatenate the validation set output of the primary learner with the test set output to obtain a set of predicted labels and predicted probabilities, and add the true labels to this set as the input of the meta-learner training set; A fractional-order Kolmogorov-Arnold network model is proposed as a meta-learner, and the back-propagation algorithm is used for parameter tuning. The tuned meta-learner is used for prediction to obtain the prediction results.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the unknown forged image detection method based on stacked generalized ensemble learning according to any one of claims 1 to 7.

10. An information data processing terminal, characterized in that: The information data processing terminal includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the unknown forged image detection method based on stacked generalized ensemble learning as described in any one of claims 1 to 7.

Images