A copyright protection method based on a customized generative model with robust watermark optimization

Abstract

This invention discloses a copyright protection method for a customized generative model based on robust watermark optimization, comprising: embedding a randomly generated initial watermark sequence into the parameters of a basic generative model to obtain an initial watermarked generative model; using the initial model as the weights of a proxy model to construct a proxy customized model to simulate the attacker's customized training process; using the proxy model to simulate fine-tuning training on a customized dataset; constructing a watermark robustness optimization objective based on gradients, and introducing a Fisher information matrix to weight and constrain the watermark update gradient, so that the watermark parameters are iteratively optimized towards the robust watermark subspace; after multiple rounds of proxy training and watermark updates, obtaining the optimized robust watermark and re-embedding it into the basic model for external release, for use in any downstream customized generative task; in the verification stage, inputting the image to be detected into a pre-trained extraction network to extract watermark information and match it with the original watermark to complete model verification and copyright verification.

Description

Technical Field

[0001] This invention belongs to the field of digital active forensics model watermarking technology, and particularly relates to a copyright protection method for a customized generation model based on robust watermark optimization. Background Technology

[0002] Currently, existing technologies for image copyright protection arising from customized training of generative models can be mainly divided into two categories: proactive defense methods based on data perturbation and passive evidence collection methods based on the model level. Data perturbation-based methods typically add imperceptible perturbations to the image to interfere with model training, making it difficult for the customized model to learn effective features. Model watermarking-based methods, on the other hand, embed watermark information within the generative model to achieve traceability and evidence collection of generated content. However, both of these methods still have significant shortcomings in practical applications.

[0003] On the one hand, proactive defense methods based on data perturbation typically rely on users to preprocess data before uploading, making it difficult to protect publicly released image data. Furthermore, they are prone to failure under compression and cropping on social media platforms. Simultaneously, their defensive effectiveness significantly decreases when adversarial examples constitute a low proportion of the training data. On the other hand, while existing model watermarking methods can achieve model ownership verification to some extent, in customized generation scenarios, attackers often have complete access to the white-box model and can perform secondary fine-tuning through methods such as DreamBooth and LoRA. This leads to severe degradation or even complete failure of the original watermark features, making reliable traceability of the generated content difficult.

[0004] Furthermore, existing model watermarking technologies mostly focus on the design of watermark embedding methods, while paying less attention to the robustness differences of watermarks within the watermark space itself. In actual customized training processes, different watermark sequences exhibit significant differences in their ability to retain data after model fine-tuning, and randomly selected watermarks often struggle to withstand the disruptions caused by model parameter updates. These issues make it difficult for existing technologies to simultaneously ensure watermark robustness and cross-scenario generalization ability in white-box customized scenarios without affecting model generation performance, thus severely limiting the effectiveness of model watermarking technology in practical copyright protection.

[0005] Therefore, how to actively select and optimize robust watermark information from the watermark space in a white-box fine-tuning environment of customized generative models, so that it can still be stably extracted in various customized training scenarios, and achieve reliable model copyright protection and content traceability, has become a key technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] Purpose of the Invention: The purpose of this invention is to overcome the shortcomings of existing model watermarks, such as easy degradation, easy failure, and poor cross-scene generalization ability during customized generation training. It provides a copyright protection method for customized generation models based on robust watermark optimization. By actively selecting and optimizing highly robust watermark information in the watermark space, the watermark can still be stably extracted during downstream customized training processes such as DreamBooth, LoRA, and Custom Diffusion. This enables reliable traceability and copyright protection of the generated content, solving the problem of insufficient robustness of existing model watermarks in white-box customized training scenarios.

[0007] Technical solution: The present invention provides a copyright protection method for a customized generation model based on robust watermark optimization, comprising the following steps:

[0008] Step 1: Use the basic generation model to be embedded with watermark as the initial model. In the watermark embedding stage, embed the randomly generated initial watermark sequence into the generation model parameters to obtain the initial watermarked generation model.

[0009] Step 2: Use the initial watermarked generation model as the initial weights of the proxy model to construct a proxy customized model for simulating the attacker's customized training process;

[0010] Step 3: Use the proxy-customized model to perform simulated training on a customized dataset to obtain the gradient information of the model during the customized training process;

[0011] Step 4: Construct a watermark robustness optimization objective based on the gradient information. By introducing the Fisher information matrix, the watermark update gradient is weighted and constrained, so that the watermark parameters are iteratively optimized towards the robust watermark subspace.

[0012] Step 5: After multiple rounds of agent training and watermark updates, the optimized robust watermark is obtained and re-embedded into the basic generation model, and then released to the public for use in any downstream customized generation task.

[0013] Step 6: In the content verification stage, the image to be detected is input into the pre-trained watermark extraction network to extract the corresponding watermark information and match it with the original watermark, thereby completing the model verification and copyright verification.

[0014] Furthermore, step 1 specifically involves: using the randomly generated binary watermark sequence Input the watermark encoding network, where n represents the length of the watermark sequence, to obtain the watermark features used for embedding; construct a watermark extraction network to recover the watermark sequence from the generated image; and use the VAE decoder in the basic generative model. To provide a watermark carrying structure, the VAE decoder is adjusted during the watermark embedding stage to ensure both image reconstruction capability and watermark extractability; an image reconstruction loss is also set during training. Watermark extraction loss :

[0015] ;

[0016] ;

[0017] in, This represents the original input image. This represents the reconstructed watermarked image. Describing the L2 norm, Represents the binary cross-entropy loss function;

[0018] by Jointly optimize relevant parameters. Let represent the hyperparameters of the watermark loss, and L represent the total embedding training loss function. Finally, the initial watermarked generation model is obtained.

[0019] Furthermore, step 2 specifically involves: using the initial watermark generation model obtained in step 1 as the initial weights of the proxy model, constructing a proxy-customized model to simulate the attacker's customized training behavior; the network structure of the proxy-customized model is consistent with the real downstream customized training strategy, including any one of DreamBooth fine-tuning, LoRA low-rank fine-tuning, or CustomDiffusion selective fine-tuning; the proxy-customized model takes the reference image and corresponding text prompts as input and executes the same training configuration as the attacker, including the learning rate, number of steps, and range of trainable modules, thereby ensuring that the parameter update direction generated by the proxy training is consistent with the real attack scenario; through the proxy-customized model, effective gradient information and sensitive direction estimation for subsequent robust watermark optimization can be obtained without relying on the details of the attacker's model.

[0020] Furthermore, step 3 specifically involves: performing simulated fine-tuning training on a customized dataset, wherein the customized dataset includes reference images of the target subject / concept and their textual descriptions; and in each round of training, constructing a customized training loss based on the noise prediction mechanism of the diffusion model. The basic diffusion loss is denoted as:

[0021] ;

[0022] in, This represents the original input image. Indicates a noise predictor. This represents the image after noise has been added. This indicates additional image description conditions. Indicates the intensity of added noise. This represents the true value of the added noise;

[0023] Introducing customized constraint terms yields:

[0024] ;

[0025] in, This indicates that the prior loss hyperparameter is used to alleviate the overfitting problem. Image data such as representation and customization of images, This indicates a description of the corresponding image;

[0026] right The gradients of the trainable parameters of the model are calculated, and each step or several steps of fine-tuning is recorded as "gradient information in the customized training process". The gradient information is used to characterize the direction of the damage to the watermark-related parameters by the customized fine-tuning, and to provide a basis for subsequent watermark robustness optimization.

[0027] Furthermore, step 4 specifically involves: constructing a watermark robustness optimization objective based on the gradient information obtained in step 3. This optimization objective aims to maximize the extractability of the watermark after customized training and to ensure that watermark updates avoid directions highly sensitive to customized training. The overall optimization objective is... for:

[0028] ;

[0029] in, Hyperparameters representing watermark loss This indicates the loss in watermark extraction;

[0030] Approximate calculation of the Fisher information matrix To characterize parameter sensitivity:

[0031] ;

[0032] in, Representing a diagonal matrix, This represents gradient information during the training process. Represents the mathematical expectation;

[0033] and utilize Apply weighted constraints to the watermark update gradient; using the current iteration watermark as... Construct an equivalent update direction function and according to step size Perform a multi-step update:

[0034] ;

[0035] in, This represents the step size, used to constrain the range of each update. Represents a signal function;

[0036] By introducing Fisher's iterative optimization, the watermark parameters are gradually converged to the robust watermark subspace, thereby improving their ability to maintain the watermark under different customized fine-tuning.

[0037] Furthermore, step 5 specifically involves: during the robust watermark optimization process, an alternating iterative strategy of surrogate fine-tuning and watermark updating is adopted: in each round, at least one step of customized fine-tuning is performed on the surrogate model to update its parameters and generate new gradient information, and then the watermark is iteratively updated multiple times according to step 4 to obtain the updated watermark. The updated watermark is re-embedded into the watermark carrier structure of the basic generation model to obtain a robust watermark generation model. The robust watermark generation model is then released to the public for use by any downstream customized generation task and for subsequent copyright verification.

[0038] Furthermore, step 6 specifically involves: during the verification phase, acquiring the image generated by the model to be detected. The image is then input into a pre-trained watermark extraction network. Output watermark extraction ; Extract watermark With the original watermark The matching is performed using Bit-Acc as the criterion. Bit-Acc is defined as follows:

[0039] ;

[0040] Where n represents the length of the watermark sequence;

[0041] When the Bit-Acc is higher than the preset threshold, the model to be detected is determined to be a robust watermarked generation model from the published model, thus completing model verification and copyright verification; when it is lower than the threshold, it is determined that the homology is not satisfied; in order to avoid the interference of image quality differences on the determination, a semantic consistency index is selected as an auxiliary quality evaluation of the generated image quality.

[0042] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of the present invention.

[0043] The present invention also discloses a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the method of the present invention.

[0044] The present invention also discloses a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method of the present invention.

[0045] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0046] 1. This invention simulates the real customized training process by constructing a proxy customization model, and optimizes the robustness of the watermark based on the gradient information of the proxy model during the customized fine-tuning process. This allows the watermark to gradually migrate from the random watermark space to the robust watermark subspace, so that the watermark can still be stably extracted after white-box customized training such as DreamBooth, LoRA, and Custom Diffusion. This effectively solves the problems of watermark degradation, failure, and poor generalization ability of existing models in customized generation scenarios.

[0047] 2. In the process of watermark optimization, this invention introduces Fisher information matrix to weight the watermark gradient update. While ensuring the stability of watermark optimization, it suppresses parameter perturbation in directions that are highly sensitive to customized training. This enables the watermark optimization process to converge to a stable and robust watermark state in a smaller number of iterations. Without affecting the original generation quality of the generation model, it significantly improves the robustness and cross-scene adaptability of the watermark.

[0048] 3. This invention does not require any active perturbation processing of user-side data. Watermark embedding and optimization are completed by the model provider. It can achieve unified copyright traceability for all customized content generated by the published model, and avoids the problem of unstable defense effect when the data perturbation method is too low or the platform is compressed. It has higher engineering practicality and promotion value. Attached Figure Description

[0049] Figure 1 This is a flowchart of the training method for the watermark embedding model provided in an embodiment of the present invention;

[0050] Figure 2 This is a training flowchart based on robust watermark optimization provided in an embodiment of the present invention;

[0051] Figure 3 This is a flowchart of the framework based on robust watermarking optimization provided in the embodiments of the present invention;

[0052] Figure 4 This is a flowchart of the customized generated model copyright protection scheme provided in the embodiments of the present invention. Detailed Implementation

[0053] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0054] Example 1

[0055] A copyright protection method based on a robust watermark optimization-based customized generative model includes:

[0056] The basic generative model embedded with the initial random watermark is used as the initial optimization model. The corresponding gradient information is obtained by simulating customized training processes such as DreamBooth, which is used for subsequent watermark robustness optimization. In this embodiment, the proxy training is carried out through multiple alternating iterations to ensure that the gradient information can truly reflect the parameter change trend of the model in the actual customized scenario.

[0057] Based on the obtained gradient information, the current watermark parameters are optimized, and the Fisher Information Matrix (FIM) is introduced during the watermark gradient update process to adaptively adjust the gradient update direction, enhance the optimization strength for the stable direction, and enable the watermark parameters to gradually converge from the random initial state to the robust watermark subspace.

[0058] The robust watermark obtained after multiple rounds of gradient optimization and Fisher information matrix constraints is re-embedded into the parameters of the basic generation model to obtain the final released robust watermark generation model.

[0059] When the robust watermark generation model is used for any downstream customized generation task, even after white-box fine-tuning training, the generated image can still stably recover the corresponding watermark information, thereby achieving copyright protection and source traceability of the generated content.

[0060] like Figure 1 As shown, in this embodiment, the watermark embedding initialization stage includes:

[0061] A watermark embedding network and a watermark extraction network are established. The watermark embedding network is used to embed watermark information into the parameters of the diffusion model, and the watermark extraction network is used to recover the corresponding watermark information from the generated image.

[0062] A pre-trained watermark decoder is used to construct a loss function for watermark information embedding and extraction. By constraining the image after watermark embedding to maintain the original visual quality, the watermark extraction network can also accurately recover the corresponding watermark sequence.

[0063] The VAE decoder of the diffusion model is trained using a training dataset, enabling it to not only possess the original image reconstruction capabilities but also to further acquire watermark embedding capabilities. In this embodiment, the VAE decoder serves as the key carrier structure for watermark embedding.

[0064] Based on the above loss function, the model parameters of the watermark embedding network and VAE decoder are jointly updated until the source image can be well reconstructed and the watermark extraction network can accurately extract the corresponding watermark information with a preset precision. After the model training is completed, the original VAE decoder of the diffusion model is replaced with the trained watermarked VAE decoder, so that the watermark can be initially embedded in the diffusion generation model.

[0065] like Figure 2 As shown, in this embodiment, the robust watermark optimization training phase includes:

[0066] The initial state for optimization is to initialize the diffusion generation model for embedding watermarks and the corresponding initial watermark information; the initial watermark is a watermark sequence that is randomly generated during the watermark embedding stage and has been embedded into the corresponding parameters of the diffusion model.

[0067] The initial watermark-embedded diffusion generation model is subjected to customized training such as DreamBooth simulation, and the gradient information of the model parameters with respect to the training loss is recorded. The obtained surrogate fine-tuning training gradient information is used to perform multi-step gradient update iterative optimization of the current watermark parameters. In each gradient update process, the Fisher Information Matrix (FIM) is introduced to enhance the optimization strength of the stable direction, thereby providing a better watermark update guidance direction.

[0068] Based on the above-mentioned gradient-based watermark multi-round iterative update, an optimized robust watermark is obtained. The optimized robust watermark is then re-embedded into the corresponding parameters of the diffusion generation model, which is used as the final released robust watermark generation model.

[0069] like Figure 3 As shown, this invention uses the initialization of the watermark-embedded generative model and its corresponding initial watermark as the starting point for optimization. It constructs a proxy watermark model to perform a one-step DreamBooth fine-tuning of the generative model and obtains the corresponding model gradient information. Based on this, the watermark is updated and optimized using the gradient information, and the updated watermark is re-embedded into the generative model to form a new watermark model. The above-mentioned "proxy fine-tuning—watermark update" process is executed iteratively in multiple rounds until a robust watermark with high extraction stability is obtained after proxy fine-tuning training, thus completing the overall optimization process of the robust watermark. Specifically:

[0070] Training parameters include:

[0071] Use the Adam optimizer and set the initial learning rate and batch size;

[0072] The PASCAL VOC dataset was used for training the watermark embedding.

[0073] The VGGFace2 dataset, commonly used for customized training, is used to train the model proxy.

[0074] The total loss function for agent fine-tuning and watermark optimization is constructed as follows:

[0075] ;

[0076] in, This represents the total loss function during the robust watermarking optimization phase. This represents the fine-tuning loss of the generative model during agent-customized training, used to constrain the generative model's ability to generate data under customized conditions. This represents the watermark constraint loss, used to constrain the extractability of watermarks in images generated after surrogate fine-tuning training; The weighting coefficient represents the watermark constraint loss, used to balance the relationship between generation performance and watermark robustness.

[0077] The customized training loss is expressed as:

[0078] ;

[0079] in, Indicated based on reference image A customized training total loss function was constructed. )

[0080] This represents the basic denoising training loss of the diffusion model under the original generation task; This represents the weight coefficient corresponding to the customized constraint term; This indicates the parameters of the model generated by the current agent watermark. The predicted noise vector; Indicates the diffusion time step The intermediate state obtained by adding noise to the training image; Represents customized textual semantic prompts corresponding to the training images; This represents the actual Gaussian noise added; This represents the square operation of the 2-norm.

[0081] The watermark constraint loss is defined as:

[0082] ;

[0083] in, Represents the binary cross-entropy loss function; This represents a pre-trained watermark decoding network; This represents the approximate generated image obtained through a single diffusion inversion prediction. This indicates the watermark information corresponding to the current iteration.

[0084] By minimizing the watermark constraint loss, the watermark can still be stably extracted from the generated image after undergoing agent-customized fine-tuning training, thereby ensuring the detectability and consistency of the watermark during multiple rounds of iterative optimization.

[0085] like Figure 3 and Figure 4 As shown, in the robust watermark optimization and actual deployment phase, the watermark to be optimized is first denoted as... and order Simultaneously, the generated model after initializing the embedded watermark is used as the initial proxy watermark model, denoted as... In the first In each iteration, the following steps are performed:

[0086] 1. Proxy model update:

[0087] The proxy watermarking model parameters obtained in the previous round are copied and used as the proxy model parameters for the current iteration. And based on customized training loss right Perform a gradient update step to obtain the updated proxy model, which is used to simulate the attacker's actions in the first step. The direction of parameter changes in DreamBooth fine-tuning. This step is used to obtain the current watermark. Below is the model's actual gradient information during the customized training process.

[0088] 2. Constructing the watermark update direction based on the Fisher information matrix:

[0089] After obtaining the parameter gradients of the proxy model, the approximate value of the Fisher information matrix related to the watermark is calculated and combined with the watermark gradient to obtain the watermark update direction function.

[0090] ;

[0091] in, This represents the equivalent gradient weighted by the agent-customized training loss and the Fisher information matrix, used to characterize the sensitivity of the current watermark to the model output during customized training.

[0092] 3. Watermark parameter update:

[0093] Use step size and symbolic functions The current watermark is updated iteratively using the following formula:

[0094] ;

[0095] in, Indicates the first Watermark parameters after round iteration; The preset watermark update step size; The sign function is used to extract the sign information of the Fisher weighted gradient, optimizing the watermark update along the main gradient direction. This update method allows the watermark to gradually move away from directions highly sensitive to customized training while converging towards a robust subspace.

[0096] 4. Practical Applications and Deployment:

[0097] The final result This is the optimized robust watermark, which can be re-embedded into the generation model for use in any downstream customized generation task. Even after multiple rounds of white-box fine-tuning, it can still maintain high watermark extraction stability.

[0098] To verify the superiority of the method proposed in this embodiment, the following mainstream customized training schemes were used for verification:

[0099] DreamBooth (High-Fidelity Subject-Driven Fine-Tuning for Text-to-ImageModels) is a high-fidelity subject-driven method for customized fine-tuning of text-to-image models; LoRA (Low-Rank Adaptation of Large Models) is a low-rank parameter adaptation method for large models; Custom Diffusion (Personalization of Diffusion Models via Parameter-Efficient Fine-Tuning).

[0100] A parameter-efficient customized training technique based on a diffusion model;

[0101] The following evaluation indicators will be used:

[0102] Bit precision (Bit-Acc): This is obtained by comparing the watermark information extracted from the customized image with the actual value of the embedded watermark. The higher the value, the better (it is more convincing in the copyright verification stage).

[0103] The Semantic Similarity Index (DINO Score) measures the similarity between two images in terms of high-level visual structure and subject representation. This index is based on semantic features extracted by a pre-trained DINO (Self-Distillation with No Labels) model. It measures semantic consistency by calculating the normalized dot product of the feature vectors of the two images; a higher score is better (indicating greater similarity in visual features). The formula for calculating the DINO Score is as follows:

[0104] ;

[0105] in, This represents the semantic feature vector of the first image extracted by the DINO model; This represents the semantic feature vector of the second image extracted by the DINO model; Represents the dot product of two vectors; and This represents the L2 norm of the corresponding vector.

[0106] Table 1. Results of different customized training methods

[0107]

[0108] As can be seen from the results listed in Table 1, this embodiment compares and evaluates the performance differences between random watermarking and the robust watermarking optimization scheme proposed in this invention under three typical customized training scenarios: DreamBooth, LoRA, and CustomDiffusion.

[0109] The first row of the table shows the model's performance under various metrics when using random watermarking; the second row shows the performance results obtained after applying the robust watermarking optimization strategy of this invention. The experimental data shows that:

[0110] 1. The watermark extraction accuracy is significantly improved, indicating that the present invention has higher robustness in customized training scenarios, with a numerical improvement of 2% to 13%.

[0111] 2. The structural consistency and fidelity of the customized image generation fluctuate very little, indicating that the present invention has almost no negative impact on the generation quality.

[0112] The above embodiments are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several improvements and equivalent substitutions without departing from the principle of the present invention. All such improvements and equivalent substitutions to the claims of the present invention fall within the protection scope of the present invention.

Claims

1. A copyright protection method for a customized generation model based on robust watermark optimization, characterized in that, Includes the following steps: Step 1: Use the basic generation model to be embedded with watermark as the initial model. In the watermark embedding stage, embed the randomly generated initial watermark sequence into the generation model parameters to obtain the initial watermarked generation model. Step 2: Use the initial watermarked generation model as the initial weights of the proxy model to construct a proxy customized model for simulating the attacker's customized training process; Step 3: Use the proxy-customized model to perform simulated training on a customized dataset to obtain the gradient information of the model during the customized training process; Step 3 specifically involves: performing simulated fine-tuning training on a customized dataset, which includes reference images of the target subject / concept and their textual descriptions; in each round of training, constructing a customized training loss based on the noise prediction mechanism of the diffusion model. The basic diffusion loss is denoted as: ； in, This represents the original input image. Indicates a noise predictor. This represents the image after noise has been added. This indicates additional image description conditions. Indicates the intensity of added noise. This represents the true value of the added noise; Introducing customized constraint terms yields: ； in, This represents the prior loss hyperparameter, used to mitigate the overfitting problem. Image data such as representation and customization of images, This indicates a description of the corresponding image; right The gradients of the trainable parameters of the model are calculated, and each step or several steps of fine-tuning are recorded as gradient information in the customized training process. The gradient information is used to characterize the direction of the damage to the watermark-related parameters by the customized fine-tuning, and to provide a basis for subsequent watermark robustness optimization. Step 4: Construct a watermark robustness optimization objective based on the gradient information. By introducing the Fisher information matrix, the watermark update gradient is weighted and constrained, so that the watermark parameters are iteratively optimized towards the robust watermark subspace. Step 4 specifically involves: constructing a watermark robustness optimization objective based on the gradient information obtained in Step 3. This objective aims to maximize the extractability of the watermark after customized training and to ensure that watermark updates avoid directions highly sensitive to customized training. The overall optimization objective is... for: ； in, Hyperparameters representing watermark loss This indicates the loss in watermark extraction; Approximate calculation of the Fisher information matrix To characterize parameter sensitivity: ； in, Representing a diagonal matrix, This represents gradient information during the training process. Express the mathematical expectation; and use Apply weighted constraints to the watermark update gradient; using the current iteration watermark as... Construct an equivalent update direction function and according to step size Perform a multi-step update: ； in, This represents the step size, used to constrain the range of each update. Represents a signal function; By introducing Fisher's iterative optimization, the watermark parameters are gradually converged to the robust watermark subspace, thereby improving their ability to maintain the watermark under different customized fine-tuning. Step 5: After multiple rounds of agent training and watermark updates, the optimized robust watermark is obtained and re-embedded into the basic generation model, and then released to the public for use in any downstream customized generation task. Step 6: In the content verification stage, the image to be detected is input into the pre-trained watermark extraction network to extract the corresponding watermark information and match it with the original watermark, thereby completing the model verification and copyright verification.

2. The copyright protection method for a customized generation model based on robust watermark optimization according to claim 1, characterized in that, Step 1 specifically involves: generating a random binary watermark sequence. Input the watermark encoding network, where n represents the length of the watermark sequence, to obtain the watermark features used for embedding; construct a watermark extraction network to recover the watermark sequence from the generated image; and use the VAE decoder in the basic generative model. To accommodate the watermark, the VAE decoder is adjusted during the watermark embedding stage to ensure watermark extractability while maintaining image reconstruction capabilities; an image reconstruction loss is also set during training. Watermark extraction loss : ； ； in, This represents the original input image. This represents the reconstructed watermarked image. Represents the L2 norm. Represents the binary cross-entropy loss function; by Jointly optimize relevant parameters. Let represent the hyperparameters of the watermark loss, and L represent the total embedding training loss function, ultimately yielding the initial watermarked generation model.

3. The copyright protection method for a customized generation model based on robust watermark optimization according to claim 1, characterized in that, Step 2 specifically involves: using the initial watermark generation model obtained in Step 1 as the initial weights of the proxy model, constructing a proxy-customized model to simulate the attacker's customized training behavior; the network structure of the proxy-customized model is consistent with the real downstream customized training strategy, including any one of DreamBooth fine-tuning, LoRA low-rank fine-tuning, or Custom Diffusion selective fine-tuning; the proxy-customized model takes the reference image and corresponding text prompts as input and executes the same training configuration as the attacker, including learning rate, number of steps, and range of trainable modules, thereby ensuring that the parameter update direction generated by the proxy training is consistent with the real attack scenario; through the proxy-customized model, effective gradient information and sensitive direction estimation for subsequent robust watermark optimization can be obtained without relying on the details of the attacker's model.

4. The copyright protection method based on a robust watermark optimization customized generation model according to claim 1, characterized in that, Step 5 specifically involves: In the robust watermark optimization process, an alternating iterative strategy of surrogate fine-tuning and watermark updating is adopted: In each round, at least one step of customized fine-tuning is performed on the surrogate model to update its parameters and generate new gradient information. Then, the watermark is iteratively updated multiple times according to step 4 to obtain the updated watermark. ; The updated watermark is re-embedded into the watermark carrier structure of the basic generation model to obtain a robust watermark generation model. The robust watermark generation model is then released to the public for use in any downstream customized generation task and for subsequent copyright verification.

5. The copyright protection method for a customized generation model based on robust watermark optimization according to claim 1, characterized in that, Step 6 specifically involves: during the verification phase, acquiring the image generated by the model to be detected. The image is then input into a pre-trained watermark extraction network. Output watermark extraction ; Extract watermark With the original watermark The matching is performed using Bit-Acc as the criterion. Bit-Acc is defined as follows: ； Where n represents the length of the watermark sequence; When the Bit-Acc is higher than the preset threshold, the model to be detected is determined to be a robust watermarked generation model from the published model, thus completing model verification and copyright verification; when it is lower than the threshold, it is determined that the homology is not satisfied; in order to avoid the interference of image quality differences on the determination, a semantic consistency index is selected as an auxiliary quality evaluation of the generated image quality.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 1.

7. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

Images