Copyright causal attribution method based on noise residual watermark and spectral diffusion model

Abstract

The invention discloses a copyright causal attribution method based on a noise residual watermark and a spectral diffusion model, which comprises the following steps: dividing a data set, generating a secret information set, generating noise residual watermark information, and adding a noise residual watermark to a picture of each concept; spectral diffusion model training is carried out, a mean square error loss function is introduced, then decoding and prediction are carried out, and model training is carried out; generating a synthetic picture, performing watermark information extraction and attribution calculation, and performing concept determination; for the condition of multiple watermarks, picture segmentation is carried out, noise residual watermarks are added to pictures, and an objective function is updated to consider the loss of a spectral diffusion model (SDM) and the loss of a mean square error (MSE). According to the method, the accuracy of image attribution is remarkably improved, extremely high flexibility and adaptability are shown, universality is achieved, an innovative solution is provided on the technical level, far-reaching influences are achieved on the law, ethics and society levels, and a solid foundation is provided for sustainable development of the GenAI technology.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology and digital watermark technology, and in particular to a copyright causal attribution method based on noise residual watermark and spectral diffusion model. Background Art

[0002] In the field of GenAI, especially in image synthesis technology based on diffusion models, how to accurately identify and attribute the concepts of training data contained in the synthesized images is a crucial issue. Existing technologies mostly use passive attribution methods based on visual similarity. These methods cannot accurately determine the impact of specific training data on the generated image, resulting in the content used by creative workers in GenAI training not being properly recognized and rewarded. In addition, these issues also involve legal and ethical issues such as copyright protection and intellectual property ownership. Therefore, the development of a technology that can accurately attribute digital copyrights has important practical value for protecting the rights and interests of creative workers and promoting the development of the creative economy.

[0003] In the application of GenAI, the diffusion model generates a target image from a noisy image through a gradual denoising process. This process involves starting from noise, denoising step by step until the target image is generated, and adjusting the parameters of the denoising network through optimization training. However, while the application of this technology brings innovation and convenience, it also raises concerns about the protection of the rights of creative workers. Since GenAI tends to imitate and copy existing successful models and styles during the learning and generation process, this not only ignores the uniqueness and innovation of individual creators, but may also lead to the convergence of collective creativity. This convergence may limit the diversity and innovation in fields such as music and art, thereby affecting the healthy development of the creative economy. In addition, the issues of discrimination and false information cannot be ignored, because AI-generated content relies on the statistical laws in its training data, and the limitations of current technology may lead to discriminatory or false information in AI-generated content.

[0004] In summary, developing a technology that can accurately attribute credit can not only protect the rights of creative workers, but also promote the diversity and innovation of the creative economy, while reducing the lack of social interaction, discrimination, and the generation of false information. This is of immeasurable value for maintaining the healthy development of the creative industry and social responsibility. Summary of the invention

[0005] In view of this, the present invention provides a copyright causal attribution method based on noise residual watermark and spectral diffusion model.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A copyright causal attribution method based on noise residual watermark and spectral diffusion model includes the following steps:

[0008] Step 1: Add a noise residual watermark

[0009] First, divide the data set, generate a secret information set, generate noise residual watermark information, and then add a noise residual watermark to the image of each concept;

[0010] Step 2: Generate model training

[0011] First, the spectral diffusion model is trained, the mean square error loss function is introduced, and then decoding and prediction are performed, and the model training is performed;

[0012] Step 3: Reasoning and Attribution

[0013] First, generate a synthetic image, extract watermark information, calculate attribution, and determine the concept;

[0014] Step 4: Multiple Watermarks

[0015] First, the image is segmented and a noise residual watermark is added to the image. For the case of multiple watermarks, the objective function is updated to consider the spectral diffusion model SDM loss and the mean square error MSE loss.

[0016] Preferably, step 1 comprises the following steps:

[0017] Step 1.1: Divide the dataset into multiple concept groups, each group corresponds to a unique concept feature;

[0018] Step 1.2: Generate a set of secret information, generate a set of orthogonal secret information for each concept, and convert the initial secret information into an orthogonal vector through the Schmidt orthogonalization process;

[0019] Step 1.3: Generate noise residual watermark information. For each concept image, use HiNet deep image hider to encrypt it and then make a difference with the original image to get the noise residual. Then calculate the average value of these residuals as the watermark information.

[0020] Step 1.4: Adding a noise residual watermark, the noise residual watermark is added to the image by transforming the function and adjusting the watermark size to fit the image resolution.

[0021] Preferably, in step 1.1, the concept features include objects, scenes, templates, themes, styles and artists.

[0022] Preferably, in step 1, the mathematical expression of generating the noise residual watermark information is:

[0023]

[0024] Among them, X i represents the i-th image, Hi(·) represents the HiNet deep image hider;

[0025] In step 1, the mathematical expression of the process of adding noise residual watermark is:

[0026]

[0027] in, is a transformation function used to add a noise residual watermark to image X to generate a watermarked image X W , m is the watermark embedding strength, and the R(·) function is used to adjust the watermark size to fit the input resolution (h, w).

[0028] Preferably, step 2 comprises the following steps:

[0029] Step 2.1: Spectral Diffusion Model (SDM) training: Use the watermarked image to train the SDM denoising module, with the goal of restoring the original image.

[0030] Step 2.2: Introduce the mean square error (MSE) loss function to help the model learn the association between the synthetic image and the training concept;

[0031] Step 2.3: Decoding and prediction, use the decoder to decode the denoised code and predict the embedded secret information;

[0032] Step 2.4: During the training process, consider the spectral diffusion model SDM loss and the mean square error MSE loss, and balance the two losses by adjusting the weights.

[0033] Preferably, in step 2, the watermarked image X is used W The denoising module ∈ of the training spectral diffusion model θ (·), the objective function is expressed as:

[0034]

[0035] Among them, ∈ is the noise added in the tth step, z t yes The noisy version of Indicates that the two-dimensional Fourier transform maps the image features into frequency domain features;

[0036] In step 2, the mean square error MSE loss function is introduced as:

[0037]

[0038] Among them, L MSE (·) is used to calculate and W j Bound Sj With the predicted The mean square error between

[0039] In step 2, decoder D is used L The denoised code Decode; the embedded secret information passes through the secret information decoder D s To make a prediction, it can be expressed as:

[0040]

[0041] In step 2, during model training, the weight α is adjusted to balance the impact of the two losses; the specific formula is:

[0042] Loss = L SDM +α·L MSE .

[0043] Preferably, step 3 comprises the following steps:

[0044] Step 3.1: Generate synthetic images, sample random noise from Gaussian distribution, input SDM to obtain latent feature vectors, and then input decoder to obtain synthetic images, which are embedded with watermark information;

[0045] Step 3.2: Watermark information extraction, extracting the embedded secret information sequence from the synthetic image;

[0046] Step 3.3: Attribution calculation: Calculate the attribution function based on the extracted secret information sequence and all secret information sequences to determine the influencing source behind the synthetic image;

[0047] Step 3.4: Concept determination, determine the concept watermark closest to the synthesized image by maximizing the attribution function.

[0048] Preferably, in step 3, the formula for attribution calculation is:

[0049]

[0050] Among them, predict the secret information sequence All secret information sequences S j , j∈{1,2,…,N};

[0051] in, is an indicator function, if the condition If true, return value 1, otherwise return 0;

[0052] In step 3, the specific expression of concept determination is:

[0053]

[0054] Preferably, step 4 comprises the following steps:

[0055] Step 4.1: Image segmentation, divide each image into two halves and adjust the size of each watermark to fit its respective half, ensuring that each half of the image carries different watermark information related to a specific concept;

[0056] Step 4.2: Add noise residual watermark to the image;

[0057] Step 4.3: For the case of multiple watermarks, update the objective function to take into account the spectral diffusion model SDM loss and the mean square error MSE loss.

[0058] Preferably, in step 4.2, the transformation function is updated as follows:

[0059]

[0060] Where, {·} represents the horizontal splicing function, X is the RGB image, {S i ,S j} is the secret information, {W i ,W j} is to generate noise residual watermark information;

[0061] In step 4.3, the updated objective function is expressed as follows:

[0062]

[0063] Among them, L SDM (·) is the spectral diffusion model loss, L MSE (·) is the mean square error function.

[0064] Compared with the prior art, the present invention has achieved the following technical effects:

[0065] (1) This invention uses a copyright causal attribution method based on noise residual watermark and spectral diffusion model to achieve accurate attribution of images synthesized by generative artificial intelligence (GenAI). This technological breakthrough significantly improves the accuracy of image attribution, thereby providing more solid copyright protection for creative workers. This protection mechanism ensures the legal use of creative content, allowing creative workers to obtain due recognition and remuneration for their contributions, thereby stimulating the vitality and innovation momentum of the creative economy.

[0066] (2) The attribution mechanism of the present invention is transparent, which enhances the interpretability of the GenAI model and enables users to have greater trust in the output of the model, which is crucial to improving the transparency of the model and the trust of users;

[0067] (3) The method of the present invention exhibits extremely high flexibility and adaptability, which makes the technical solution of the present invention not only limited to specific application scenarios, but also has wide applicability;

[0068] (4) The present invention effectively reduces the uncertainty of decision-making and enhances the reliability of decision-making by providing accurate attribution information;

[0069] (5) This invention takes full account of ethics and social responsibility in its design. By ensuring the legal use of creative content, it respects the rights and interests of creative workers and promotes the healthy development of GenAI technology.

[0070] (6) The present invention is also universal and can be extended to other types of artificial intelligence models and applications, which makes it possible to solve similar attribution problems in a wider range of fields; at the legal and ethical level, it provides a technical means to help solve legal and ethical issues related to GenAI, and provides strong technical support for lawmaking and ethical review.

[0071] In summary, this invention not only provides innovative solutions at the technical level, but also has far-reaching impacts at the legal, ethical, and social levels, providing a solid foundation for the sustainable development of GenAI technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is a technical framework diagram of the present invention;

[0073] Figure 2 This is a diagram showing the effect of the present invention. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] Embodiment 1:

[0076] The present invention discloses a copyright causal attribution method based on noise residual watermark and spectral diffusion model, comprising the following steps:

[0077] Step 1: Add a noise residual watermark

[0078] Step 1.1: Divide the dataset D into N concept groups, each of which corresponds to a unique concept feature; that is, C = {C 1 ,C 2 ,…,CN}.

[0079] The concept in this invention refers to specific elements or features contained in the input image when training the generative artificial intelligence (GenAI) model. These can be objects (such as cats, dogs, cars), scenes (such as beaches, city streets, forests), templates (such as resume templates, poster designs), themes (such as floral patterns, geometric figures), styles (such as Impressionism, Cubism, Surrealism) and artists (such as Van Gogh's starry sky style, Picasso's Cubist style), etc.

[0080] Step 1.2: Generate a secret information set. For each concept, generate a secret information set S = {S 1 ,S 2 ,…S j ,…S N}, where j∈{0,1,2,…,N}. Any two elements in these secret messages, S i and S j Have orthogonality;

[0081] Specifically, first initialize the secret information set S, and then use the Schmidt orthogonalization process to convert the initial

[0082] The initialization secret information set vector is converted into a set of orthogonal vectors. This set of orthogonal vectors is the generated secret information set S = {S 1 ,S 2 ,…S j ,…S N}.

[0083] Step 1.3: Generate noise residual watermark information, for each concept C j , according to its corresponding secret information S j And the image in the concept generates a unique noise residual watermark W j , where j∈{0,1,2,…,N}, W j The b-dimensional bit sequence S j ={p j1 ,p j2 ,…,p jb}Generate, where S j The elements in are either 0 or 1.

[0084] Specifically, in the jth group of concept image dataset C j Randomly select p pictures from the dataset, and encrypt each picture through the pre-trained HiNet deep image hider Hi(·) to obtain the ciphertext. Then, each original picture is subtracted from each ciphertext to obtain the noise residual, which is used to generate the noise residual watermark information; the noise residual watermark information is the average of the p noise residuals;

[0085] The mathematical expression for generating noise residual watermark information is:

[0086]

[0087] Among them, X i represents the i-th image, Hi(·) represents the HiNet deep image hider;

[0088] Step 1.4: Adding noise residual watermark, adding the noise residual watermark to the image, through the transformation function and adjusting the watermark size to adapt to the image resolution;

[0089] The mathematical expression of the process of adding noise residual watermark is:

[0090]

[0091] in, is a transformation function used to add a noise residual watermark to image X to generate a watermarked image X W , m is the watermark embedding strength, and the R(·) function is used to adjust the watermark size to fit the input resolution (h, w).

[0092] Step 2: Generate model training

[0093] Step 2.1: Spectral Diffusion Model SDM training, using watermarked image X W The denoising module ∈ of the training spectral diffusion model θ (·), the objective function is expressed as:

[0094]

[0095] Among them, ∈ is the noise added in the tth step, z t yes The noisy version of Indicates that the two-dimensional Fourier transform maps the image features into frequency domain features;

[0096] Step 2.2: Introduce the mean square error (MSE) loss function to help the model learn the association between the synthetic image and the training concept;

[0097] By introducing the mean square error MSE loss function in the spectral diffusion model training:

[0098]

[0099] Among them, L MSE (·) is used to calculate and W j Bound S j With the predicted The mean square error between

[0100] Step 2.3: Decoding and prediction, use the decoder to decode the denoised code and predict the embedded secret information;

[0101] Using decoder D L The denoised code Decode; the embedded secret information passes through the secret information decoder D s To make a prediction, it can be expressed as:

[0102]

[0103] Step 2.4: During the training process, consider the spectral diffusion model SDM loss and the mean square error MSE loss, and balance the impact of these two losses by adjusting the weight α; the specific formula is:

[0104] Loss = L SDM +α·L MSE .

[0105] Step 3: Reasoning and Attribution

[0106] Step 3.1: Generate synthetic images, first from Gaussian distribution Sample multiple random noises in the spectral diffusion model, input these random noises into the spectral diffusion model to obtain high-quality latent feature vectors, and then input the obtained latent feature vectors into the decoder D L The synthetic image X with embedded watermark information is obtained S The spectral diffusion model embeds watermark information in the images when generating these synthetic images, where each watermark is mapped to a unique orthogonal bit sequence associated with a specific training concept as a hidden signature.

[0107] Step 3.2: Watermark information extraction, using secret information decoder D S From the synthetic image X S Extract the embedded secret information sequence

[0108] Step 3.3: Attribution calculation, based on the obtained predicted secret information sequence

[0109] And all secret information sequences S j , j∈{1,2,…,N}; the formula for attribution calculation is:

[0110] in, is an indicator function, if the condition If true, return value 1, otherwise return 0;

[0111]

[0112] Among them, predict the secret information sequence All secret information sequences S j , j∈{1,2,…,N};

[0113] Step 3.4: Concept identification by maximizing the attribution function To determine the conceptual watermark that is closest to the composite image;

[0114] The specific expression of concept determination is:

[0115]

[0116] Step 4: Multiple Watermarks

[0117] Step 4.1: Image segmentation, divide each image into two halves and adjust the size of each watermark to fit its respective half, ensuring that each half of the image carries different watermark information related to a specific concept;

[0118] Step 4.2: Add noise residual watermark to the image;

[0119] Assuming the input RGB image is X, through steps 1.1 to 1.3, generate the secret information {S i ,S j}, then according to {S i ,S j}Generate noise residual watermark information {W i ,W j}.

[0120] The transformation function is updated to:

[0121]

[0122] Where, {·} represents the horizontal splicing function, X is the RGB image, {S i ,S j} is the secret information, {W i ,W j} is to generate noise residual watermark information;

[0123] Step 4.3: For the case of multiple watermarks, update the objective function to take into account the spectral diffusion model SDM loss and the mean square error MSE loss. The updated objective function is expressed as follows:

[0124]

[0125] Among them, L SDM (·) is the spectral diffusion model loss, L MSE (·) is the mean square error function.

[0126] The above description is only a preferred embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any slight modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A copyright causal attribution method based on noise residual watermark and spectral diffusion model, characterized in that: The steps include: Step 1: Add a noise residual watermark First, divide the data set, generate a secret information set, generate noise residual watermark information, and then add a noise residual watermark to the image of each concept; Step 2: Generate model training First, the spectral diffusion model is trained, the mean square error loss function is introduced, and then decoding and prediction are performed, and the model training is performed; Step 3: Reasoning and Attribution First, generate a synthetic image, extract watermark information, calculate attribution, and determine the concept; Step 4: Multiple Watermarks First, the image is segmented and a noise residual watermark is added to the image. For the case of multiple watermarks, the objective function is updated to consider the spectral diffusion model SDM loss and the mean square error MSE loss.

2. According to claim 1, a copyright causal attribution method based on noise residual watermark and spectral diffusion model is characterized in that: The step 1 comprises the following steps: Step 1.1: Divide the dataset into multiple concept groups, each group corresponds to a unique concept feature; Step 1.2: Generate a set of secret information, generate a set of orthogonal secret information for each concept, and convert the initial secret information into an orthogonal vector through the Schmidt orthogonalization process; Step 1.3: Generate noise residual watermark information. For each concept image, use HiNet deep image hider to encrypt it and then subtract it from the original image to get the noise residual. Then calculate the average value of these residuals as the watermark information. Step 1.4: Adding a noise residual watermark, the noise residual watermark is added to the image by transforming the function and adjusting the watermark size to fit the image resolution.

3. According to claim 2, a copyright causal attribution method based on noise residual watermark and spectral diffusion model is characterized in that: In step 1.1, the concept features include objects, scenes, templates, themes, styles, and artists.

4. According to claim 1, a copyright causal attribution method based on noise residual watermark and spectral diffusion model is characterized in that: In step 1, the mathematical expression of generating noise residual watermark information is: Among them, X i represents the i-th image, Hi(·) represents the HiNet deep image hider; In step 1, the mathematical expression of the process of adding noise residual watermark is: in, is a transformation function used to add a noise residual watermark to image X to generate a watermarked image X W , m is the watermark embedding strength, and the R(·) function is used to adjust the watermark size to fit the input resolution (h, w).

5. According to claim 1, a copyright causal attribution method based on noise residual watermark and spectral diffusion model is characterized in that: The step 2 comprises the following steps: Step 2.1: Spectral Diffusion Model (SDM) training: Use the watermarked image to train the SDM denoising module, with the goal of restoring the original image. Step 2.2: Introduce the mean square error (MSE) loss function to help the model learn the association between the synthetic image and the training concept; Step 2.3: Decoding and prediction, use the decoder to decode the denoised code and predict the embedded secret information; Step 2.4: During the training process, consider the spectral diffusion model SDM loss and the mean square error MSE loss, and balance the two losses by adjusting the weights.

6. According to claim 1, a copyright causal attribution method based on noise residual watermark and spectral diffusion model is characterized in that: In step 2, use the watermarked image X W The denoising module ∈ of the training spectral diffusion model θ (·), the objective function is expressed as: Among them, ∈ is the noise added in the tth step, z t yes The noisy version of Indicates that the two-dimensional Fourier transform maps the image features into frequency domain features; In step 2, the mean square error MSE loss function is introduced as: Among them, L MSE (·) is used to calculate and W j Bound S j With the predicted The mean square error between In step 2, decoder D is used L The denoised code Decode; the embedded secret information passes through the secret information decoder D s To make a prediction, it can be expressed as: In step 2, during model training, the weight α is adjusted to balance the impact of the two losses; the specific formula is: Loss=L SDM +α·L MSE 。 7. The copyright causal attribution method based on noise residual watermark and spectral diffusion model according to claim 1 is characterized in that: The step 3 comprises the following steps: Step 3.1: Generate synthetic images, sample random noise from Gaussian distribution, input SDM to obtain latent feature vectors, and then input decoder to obtain synthetic images, which are embedded with watermark information; Step 3.2: Watermark information extraction, extracting the embedded secret information sequence from the synthetic image; Step 3.3: Attribution calculation: Calculate the attribution function based on the extracted secret information sequence and all secret information sequences to determine the influencing source behind the synthetic image; Step 3.4: Concept determination, determine the concept watermark closest to the synthesized image by maximizing the attribution function.

8. The copyright causal attribution method based on noise residual watermark and spectral diffusion model according to claim 1 is characterized in that: In step 3, the formula for attribution calculation is: Among them, predict the secret information sequence All secret information sequences S j , j∈{1,2,…,N}; in, is an indicator function, if the condition If true, return value 1, otherwise return 0; In step 3, the specific expression of concept determination is:

9. The method of image processing and copyright attribution using noise residual watermark and spectral diffusion model according to claim 1, characterized in that: The step 4 comprises the following steps: Step 4.1: Image segmentation, divide each image into two halves and adjust the size of each watermark to fit its respective half, ensuring that each half of the image carries different watermark information related to a specific concept; Step 4.2: Add noise residual watermark to the image; Step 4.3: For the case of multiple watermarks, update the objective function to take into account the spectral diffusion model SDM loss and the mean square error MSE loss.

10. The copyright causal attribution method based on noise residual watermark and spectral diffusion model according to claim 9 is characterized in that: In step 4.2, the transformation function is updated as follows: Where, {·} represents the horizontal splicing function, X is the RGB image, {S i ,S j } is the secret information, {W i ,W j } is to generate noise residual watermark information; In step 4.3, the updated objective function is expressed as follows: Among them, L SDM (·) is the spectral diffusion model loss, L MSE (·) is the mean square error function.

Images