AIGC detection method with continuous learning ability and electronic equipment

Abstract

The invention discloses an AIGC detection method with continuous learning ability and electronic equipment. The method comprises the following steps: acquiring image data to construct a training data set; constructing a neural network model, and carrying out continuous learning training on the neural network model; and inputting a to-be-detected image into the neural network model subjected to continuous learning training to obtain an AIGC image detection result of the to-be-detected image. According to the method, through a continuous learning mechanism, model parameters can be automatically updated in the detection process, the method adapts to novel AIGC content, and the detection accuracy and generalization ability are improved; the whole model does not need to be retrained frequently or a large-scale new training set does not need to be constructed, and system maintenance cost and data preparation difficulty are reduced; therefore, the detection system can stably resist the constantly changing AIGC technology for a long time, and the sustainable upgrading and optimization of the detection capability are realized.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning, and particularly relates to an AIGC detection method and electronic device with continuous learning capabilities. Background Technology

[0002] With the rapid development of generative artificial intelligence (AI) technology, AI-generated images are widely used in creative and entertainment fields. However, the security detection of these generated images has become an urgent need. Existing AI-generated image detection methods mostly employ supervised learning, using pre-trained neural networks (such as CNNs and Transformers) for fine-tuning in binary classification to distinguish between real and fake images. However, traditional detection models are prone to overfitting training data, resulting in insufficient generalization ability in real-world scenarios. This is especially true when encountering fake images generated by new generative AI models, where traditional models often struggle to adapt, leading to a significant performance drop. Furthermore, simply performing multi-task training or incremental training on the model can cause a catastrophic forgetting problem, causing the model to forget previously learned features that distinguish between real and fake images, thus affecting detection performance. Therefore, a new detection method is urgently needed that can continuously learn new features that distinguish between real and fake images while constantly introducing new datasets, without forgetting its ability to differentiate between real and fake images from previous data. Summary of the Invention

[0003] In order to solve the above-mentioned technical problems, this invention proposes an AIGC detection method and electronic device with continuous learning capabilities.

[0004] In a first aspect, the present invention provides an AIGC image detection method with continuous learning capability, characterized in that the method includes the following steps:

[0005] Step S1: Obtain image data to construct a training dataset;

[0006] Step S2: Construct a neural network model and continuously learn and train it;

[0007] Step S21: Perform singular value decomposition on the weight matrix of the linear layer in the neural network model, decomposing it into a principal subspace and a residual subspace. The principal subspace corresponds to the weight part of the weight matrix composed of the first r largest singular values ​​and their corresponding left and right singular matrices, where r > 0, and is used to preserve the model's ability to distinguish between real and fake images in the previous data. The residual subspace corresponds to the weight part composed of the remaining singular values ​​and their corresponding left and right singular matrices, and is used to learn the feature patterns of real and fake images in the current dataset.

[0008] Step S22: Fix the parameters of the principal subspace and update the parameters only in the residual subspace to learn new real and fake image features;

[0009] Step S23: Based on the classification loss function, add orthogonality preservation constraints to maintain the orthogonality between the principal subspace and the residual subspace, and singular value norm constraints to control the overall norm change of the parameters before and after optimization, to form the total loss function and train the model.

[0010] Step S24: Add the principal subspace parameters obtained after training to the updated residual subspace parameters to obtain the updated linear layer weight matrix for the next task.

[0011] Step S25: Repeat steps S21 to S24 to perform continuous learning sequentially on multiple datasets;

[0012] Step S3: Input the image to be detected into the neural network model that has been continuously trained to obtain the AIGC image detection result of the image to be detected.

[0013] Preferably, in step S21, the linear layer is a linear layer contained in the self-attention module of the neural network model, including a key projection layer (k_proj), a query projection layer (q_proj), a value projection layer (v_proj), and an output projection layer (out_proj).

[0014] Preferably, in step S21, the calculation formula for the singular value decomposition is: in, The linear layer weight matrix, It is a left singular vector matrix. It is a right singular vector matrix. It is a diagonal matrix containing singular values.

[0015] Preferably, in step S21, the principal subspace weights The calculation formula is: in, It is a diagonal matrix containing the first r largest singular values. and These are the corresponding left singular vector matrix and right singular vector matrix, respectively.

[0016] Preferably, in step S21, the residual subspace weights The calculation formula is: in, It is a diagonal matrix containing the remaining nr singular values. and These are the corresponding left singular vector matrix and right singular vector matrix.

[0017] Preferably, in step S23, the orthogonality preservation constraint This is achieved through orthogonality preservation loss, and its calculation formula is as follows: in, For the left singular vector matrix of the principal subspace Left singular vector matrix of residual subspace It is assembled by columns; For the right singular vector matrix of the principal subspace With the right singular vector matrix of the residual subspace It is assembled by columns; It is an identity matrix.

[0018] Preferably, in step S23, the singular value norm constraint This is achieved through singular value norm loss, and its calculation formula is as follows:

[0019]

[0020] in, This is the optimized linear layer weight matrix. This is the linear layer weight matrix before optimization. Represents the square of the Frobenius norm. This represents the function that takes the absolute value.

[0021] Preferably, in step S23, the classification loss function For binary cross-entropy loss, the total loss function is... The calculation formula is: in, , These are hyperparameters, and all are greater than 0; This indicates an orthogonality preservation constraint; This represents the singular value norm constraint.

[0022] Preferably, in step S24, the formula for calculating the addition of parameters is: in, This is the updated linear layer weight matrix used for learning the next task. The principal subspace weights are fixed and unchanging for the current training task. These are the residual subspace weights updated after training for the current task.

[0023] In a second aspect, the present invention provides an AIGC image detection system for implementing the method, comprising:

[0024] The data acquisition module is responsible for acquiring the image to be detected;

[0025] The AIGC image detection module is responsible for inputting the image to be detected into a neural network model that has been continuously trained to obtain the AIGC image detection result of the image to be detected.

[0026] The beneficial effects of the present invention are: (1) Through the continuous learning mechanism, the model parameters can be automatically updated during the detection process to adapt to new AIGC content and improve detection accuracy and generalization ability; (2) It does not require frequent retraining of the entire model or construction of a large-scale new training set, which reduces the system maintenance cost and data preparation difficulty; (3) It enables the detection system to resist the constantly evolving AIGC technology in a long-term stable manner, and realizes the continuous upgrading and optimization of detection capabilities. Attached Figure Description

[0027] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of an AIGC detection method with continuous learning capability provided in an embodiment of the present invention.

[0029] Figure 2 This is a flowchart of the continuous learning and training process of the neural network model provided in the embodiments of the present invention.

[0030] Figure 3 This is a schematic diagram illustrating the principle of the continuous learning and training process of the neural network model provided in this embodiment of the invention. Detailed Implementation

[0031] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0032] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.

[0033] This embodiment provides an AIGC detection method with continuous learning capabilities. The core idea is to perform Singular Value Decomposition (SVD) on the weights of specific linear layers of the model, explicitly dividing the model's feature space into two mutually orthogonal subspaces: a principal subspace and a residual subspace. The principal subspace corresponds to the larger singular value portion of the weight matrix and is used to preserve the model's ability to distinguish between genuine and fake images. The residual subspace corresponds to the smaller singular value portion and is used specifically to learn and characterize the feature patterns of genuine and fake images. Specifically, continuous learning is achieved through the following steps (see Appendix). Figure 1 :

[0034] Step S1: Obtain two-dimensional image data to construct a training dataset. The training dataset includes real image samples and AIGC-generated image samples, and both types of samples are labeled. The real image samples are obtained from public image databases (such as FaceForensic++, WildDeepfake, etc.) and other legitimate channels such as real-shot collection. The AIGC-generated image samples are generated through mainstream AIGC image generation models (such as Stable Diffusion, Midjourney, DALL·E, etc.), covering images with different scenes (such as landscapes and people), different resolutions, and different generation parameters.

[0035] Step S2: Construct a neural network model and continuously train it; this embodiment uses CLIP-ViT-L / 14 (i.e., clip-vit-large-patch14). CLIP-ViT-L / 14 is a variant of the Contrastive Language-Image Pre-training model, employing a Vision Transformer architecture as the image encoder and a masked self-attention Transformer as the text encoder. See Appendix. Figure 2-3 Specifically:

[0036] Step S21: Perform singular value decomposition on the weight matrix of the linear layer in the neural network model, decomposing it into a principal subspace and a residual subspace. The principal subspace corresponds to the weight part of the weight matrix composed of the first r largest singular values ​​and their corresponding left and right singular matrices, which is used to preserve the model's ability to distinguish between real and fake images in the previous data. The residual subspace corresponds to the weight part composed of the remaining singular values ​​and their corresponding left and right singular matrices, which is used to learn the key feature patterns of real and fake images in the current dataset.

[0037] Step S22: Fix the parameters of the principal subspace and update the parameters only in the residual subspace to learn new features of real and fake images; during training, the weights of the principal subspace maintain their original values ​​to avoid damaging the previous ability to distinguish between real and fake images; only the weights of the residual subspace participate in gradient updates to learn new features of real and fake images.

[0038] Step S23: Based on the classification loss function, add orthogonality preservation constraints to maintain the orthogonality between the principal subspace and the residual subspace, and singular value norm constraints to control the overall norm change of the parameters before and after optimization, to form the total loss function and train the model. The above orthogonality preservation constraints are used to ensure that the principal subspace and the residual subspace remain strictly orthogonal; the singular value norm constraints are used to control the degree of overall norm change between the principal subspace and the residual subspace, preventing excessive drift of learned knowledge during training.

[0039] Step S24: Add the principal subspace parameters obtained after training to the updated residual subspace parameters to obtain the updated linear layer weight matrix for the next task.

[0040] Step S25: Repeat steps S21 to S24 to perform continuous learning sequentially on multiple datasets;

[0041] Step S3: Input the image to be detected into the neural network model that has been continuously trained to obtain the AIGC image detection result of the image to be detected, that is, determine whether the input image is generated by the generative artificial intelligence model.

[0042] In one embodiment, in step S21, the specific linear layer is selected from all the linear layers in the self-attention modules of the neural network model, including a projection layer (k_proj), a query projection layer (q_proj), a value projection layer (v_proj), and an output projection layer (out_proj). The k_proj, q_proj, and v_proj layers are used for modeling the self-attention mechanism of the input features, while the out_proj layer is used to enhance the nonlinearity of the features after the self-attention mechanism modeling.

[0043] The formula for calculating the singular value decomposition is as follows:

[0044] ,

[0045] in, The linear layer weight matrix, It is a left singular vector matrix. It is a right singular vector matrix. It is a diagonal matrix containing singular values ​​(the singular values ​​are arranged in descending order), where n represents the dimension of the linear layer matrix.

[0046] The formula for calculating the master subspace is as follows:

[0047] ,

[0048] in, The weights of the main subspace are determined by selecting a diagonal matrix containing the top-r largest singular values. And the corresponding left singular vector matrix and right singular vector matrix. and Obtain the master subspace weights .

[0049] The formula for calculating the residual subspace is as follows:

[0050] ,

[0051] in, The weights are for the residual subspace. This is achieved by selecting the remaining diagonal matrix containing nr singular values. And the corresponding left singular vector matrix and right singular vector matrix. and The residual subspace weights are obtained. .

[0052] In one embodiment, in step S23, the orthogonality preservation loss constraint is used to ensure that the principal subspace and the residual subspace remain strictly orthogonal, and the calculation formula is as follows:

[0053] ,

[0054] in, for and It is assembled along the column direction. Similarly... for and It is assembled by splicing along the column direction. It is an identity matrix.

[0055] The singular value norm loss constraint is used to control the degree of change in the overall norm of the parameters before and after optimization, preventing excessive drift of learned knowledge during training. The calculation formula is as follows:

[0056] ,

[0057] in, Represents the original weights The optimized weights.

[0058] The classification loss is calculated using the following formula:

[0059] ,

[0060] in, For predicting categories, For the real category, This is the binary cross-entropy loss function.

[0061] Therefore, the overall loss calculation formula is as follows:

[0062] ,

[0063] in, , These are hyperparameters, and all are greater than 0.

[0064] In one embodiment, the formula for calculating the addition of parameters in step S24 is as follows:

[0065] ,

[0066] in, This is the updated linear layer weight matrix (i.e., the k_proj, q_proj, v_proj, out_proj layer weight matrices) used for learning the next task. The master subspace weights are fixed. These are the residual subspace weights updated after training for the current task. The merged new parameters are used for singular value decomposition in the next task, retaining knowledge from the current task while learning knowledge from the next task.

[0067] In one embodiment, in step S25, steps S21 to S24 are repeated until training is completed on all datasets. The parameter change process is as follows:

[0068] ,

[0069] in, After training for the previous mission The master subspace after singular value decomposition, These are the residual subspace weights obtained after optimization in the current task.

[0070] In one embodiment, in step S3, the image to be detected is... Input the trained neural network To determine whether an input image was generated by a generative artificial intelligence model, the discriminant function is:

[0071] ,

[0072] in, ( ) is a binary discriminant function. It is the Sigmoid activation function. The threshold value is used.

[0073] This embodiment also provides an AIGC image detection system for implementing the above method, including:

[0074] The data acquisition module is responsible for acquiring the image to be detected;

[0075] The AIGC image detection module is responsible for inputting the image to be detected into a neural network model that has been continuously trained to obtain the AIGC image detection result of the image to be detected.

[0076] This embodiment also provides an electronic device, including a memory and a processor. The memory stores executable code, and when the executable code is executed by the processor, the above-described method is performed.

[0077] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. An AIGC image detection method with continuous learning capability, characterized in that, The method includes the following steps: Step S1: Obtain image data to construct a training dataset; Step S2: Construct a neural network model and continuously learn and train it; Step S21: Perform singular value decomposition on the weight matrix of the linear layer in the neural network model, decomposing it into a principal subspace and a residual subspace. The principal subspace corresponds to the weight part of the weight matrix composed of the first r largest singular values ​​and their corresponding left and right singular matrices, which is used to preserve the model's ability to distinguish between real and fake images in the previous data. The residual subspace corresponds to the weight part composed of the remaining singular values ​​and their corresponding left and right singular matrices, which is used to learn the feature patterns of real and fake images in the current dataset. Step S22: Fix the parameters of the principal subspace and update the parameters only in the residual subspace to learn new real and fake image features; Step S23: Based on the classification loss function, add orthogonality preservation constraints to maintain the orthogonality between the principal subspace and the residual subspace, and singular value norm constraints to control the overall norm change of the parameters before and after optimization, to form the total loss function and train the model. Step S24: Add the principal subspace parameters obtained after training to the updated residual subspace parameters to obtain the updated linear layer weight matrix for the next task. Step S25: Repeat steps S21 to S24 to continue learning; Step S3: Input the image to be detected into the neural network model that has been continuously trained to obtain the AIGC image detection result of the image to be detected.

2. The method according to claim 1, characterized in that, In step S21, the linear layer is a linear layer contained in the self-attention module of the neural network model, including a key projection layer, a query projection layer, a value projection layer, and an output projection layer.

3. The method according to claim 1, characterized in that, In step S21, the formula for calculating the singular value decomposition is: in, The linear layer weight matrix, It is a left singular vector matrix. It is a right singular vector matrix. It is a diagonal matrix containing singular values.

4. The AIGC detection method with continuous learning capability according to claim 3, characterized in that, In step S21, the principal subspace weights The calculation formula is: in, It is a diagonal matrix containing the first r largest singular values. and These are the corresponding left singular vector matrix and right singular vector matrix, respectively.

5. The method according to claim 4, characterized in that, In step S21, the residual subspace weights The calculation formula is: in, It is a diagonal matrix containing the remaining nr singular values. and These are the corresponding left singular vector matrix and right singular vector matrix.

6. The method according to claim 1, characterized in that, In step S23, the orthogonality preservation constraint This is achieved through orthogonality preservation loss, and its calculation formula is as follows: in, For the left singular vector matrix of the principal subspace Left singular vector matrix of residual subspace It is assembled by columns; For the right singular vector matrix of the principal subspace With the right singular vector matrix of the residual subspace It is assembled by columns; It is an identity matrix.

7. The method according to claim 1, characterized in that, In step S23, the singular value norm constraint This is achieved through singular value norm loss, and its calculation formula is as follows: in, This is the optimized linear layer weight matrix. This is the linear layer weight matrix before optimization. Denotes the square of the Frobenius norm. This represents the function that takes the absolute value.

8. The method according to claim 1, characterized in that, In step S23, the classification loss function For binary cross-entropy loss, the total loss function is... The calculation formula is: in, , These are hyperparameters, and all are greater than 0; This indicates an orthogonality preservation constraint; This represents the singular value norm constraint.

9. The method according to claim 1, characterized in that, In step S24, the formula for calculating the addition of parameters is: in, This is the updated linear layer weight matrix used for learning the next task. These are the fixed principal subspace weights in the current training task. These are the residual subspace weights updated after training for the current task.

10. An AIGC image detection system implementing the method of any one of claims 1 to 9, characterized in that, include: The data acquisition module is responsible for acquiring the image to be detected; The AIGC image detection module is responsible for inputting the image to be detected into a neural network model that has been continuously trained to obtain the AIGC image detection result of the image to be detected.

Images