Method for training watermark image generation model, watermark image generation method and device

Abstract

The present disclosure provides a watermark image generation model training method, a watermark image generation method and equipment, comprising: obtaining a training data set; inputting an original image into a variational autoencoder to output original image features; inputting original watermark information and original image features into an initial watermark encoding and decoding module, training the initial watermark encoding and decoding module until a first target loss function converges, and obtaining a watermark encoding and decoding module; inputting the original watermark information into the watermark encoding and decoding module to obtain watermark features; inputting the original watermark information into a watermark feature mapping module to obtain mapping features; inputting the watermark features and the mapping features into an adaptive attention module in an initial first image information generation network to obtain initial adaptive features, training the initial first image information generation network using the initial adaptive features until a second target loss function converges, and obtaining a watermark image generation model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image processing, and in particular to a training method of a watermark image generation model, a watermark image generation method and equipment. BACKGROUND

[0002] In recent years, diffusion models have made rapid progress, making it more convenient to generate high-quality images, but at the same time, it has raised the issue of copyright protection of generated images. Existing methods usually secretly embed watermarks in the diffusion process to improve the robustness of the watermark. However, current watermarking schemes based on the diffusion process mostly focus on embedding fixed watermarks, which to some extent limits the flexibility and personalization of the watermark, and cannot meet the user's demand for watermark diversification. For example, artists may want to embed different identifiers or signatures in different works to reflect the uniqueness of their creation and copyright information, and the existing fixed watermarking scheme obviously cannot meet this demand. In addition, although these schemes have achieved certain results in improving the robustness of the watermark, there is still a risk that malicious users will replace the decoder to evade watermark detection. Once malicious users master the cracking method of a certain decoder, they can easily remove or tamper with the watermark embedded in the image, thereby infringing the copyright of the original creator.

[0003] To address these issues, future research needs to explore more flexible and variable watermark embedding methods to achieve personalized customization of watermarks. At the same time, it is also necessary to develop more secure and reliable decoder verification mechanisms to prevent malicious users from evading watermark detection by replacing the decoder. For example, adaptive watermark embedding algorithms based on deep learning can be studied to automatically generate personalized watermark patterns according to image content and user needs and embed them into the generation process of the diffusion model. In addition, distributed ledger technologies such as blockchain can be introduced to record the embedding and verification process of the watermark to ensure the non-tamperability and traceability of the watermark. SUMMARY

[0004] Therefore, the purpose of the present disclosure is to provide a training method of a watermark image generation model, a watermark image generation method and equipment to solve or partially solve the above problems.

[0005] To achieve the above purpose, the first aspect of the present disclosure provides a training method of a watermark image generation model, the watermark image generation model comprising a watermark encoding and decoding module, a watermark feature mapping module and a first image information generation network, wherein the first image information generation network comprises an adaptive attention module, the method comprising:

[0006] obtaining a training data set, wherein the training data set comprises original watermark information and an original image;

[0007] The original image is input into the variational autoencoder, processed via the variational autoencoder, and the original image features are output;

[0008] The original watermark information and the original image features are input into the initial watermark encoding and decoding module, the original watermark information and the original image features are used to train the initial watermark encoding and decoding module, until the first target loss function corresponding to the initial watermark encoding and decoding module converges, and the watermark encoding and decoding module is obtained;

[0009] The original watermark information is input into the watermark encoding and decoding module to obtain watermark features;

[0010] The original watermark information is input into the watermark feature mapping module to obtain mapping features;

[0011] The watermark features and the mapping features are input into the adaptive attention module in the initial first image information generation network, processed via the adaptive attention module, and initial adaptive features are obtained, the initial adaptive features are used to train the initial first image information generation network, until the second target loss function corresponding to the initial first image information generation network converges, the first image information generation network training is completed, and a watermark image generation model is obtained.

[0012] Based on the above purpose, the second aspect of the present disclosure provides a watermark image generation method, applying a watermark image generation model, the method comprising:

[0013] Obtaining a to-be-processed image and target watermark information;

[0014] The to-be-processed image and the target watermark information are input into the watermark image generation model, processed via the watermark image generation model, and a watermark image is obtained, wherein the watermark image contains the target watermark information.

[0015] Based on the same inventive concept, the third aspect of the present disclosure provides a training device of a watermark image generation model, the watermark image generation model comprising a watermark encoding and decoding module, a watermark feature mapping module and a first image information generation network, the first image information generation network comprising an adaptive attention module comprising:

[0016] The data acquisition module is configured to acquire a training data set, wherein the training data set comprises original watermark information and original images;

[0017] The original image feature determination module is configured to input the original image into the variational autoencoder, process via the variational autoencoder, and output the original image features;

[0018] The first training module is configured to input the original watermark information and the original image feature into the initial watermark coding and decoding module, train the initial watermark coding and decoding module by using the original watermark information and the original image feature, until a first target loss function corresponding to the initial watermark coding and decoding module converges, and obtain the watermark coding and decoding module.

[0019] The watermark feature determination module is configured to input the original watermark information into the watermark coding and decoding module to obtain a watermark feature.

[0020] The mapping feature determination module is configured to input the original watermark information into the watermark feature mapping module to obtain a mapping feature.

[0021] The second training module is configured to input the watermark feature and the mapping feature into an adaptive attention module in the initial first image information generation network, obtain an initial adaptive feature through processing of the adaptive attention module, train the initial first image information generation network by using the initial adaptive feature, until a second target loss function corresponding to the initial first image information generation network converges, the first image information generation network is trained, and a watermark image generation model is obtained.

[0022] Based on the same inventive concept, the fourth aspect of the present disclosure provides a watermark image generation device, comprising:

[0023] The data acquisition module is configured to acquire a to-be-processed image and target watermark information.

[0024] The watermark image generation module is configured to input the to-be-processed image and the target watermark information into the watermark image generation model, process through the watermark image generation model, and obtain a watermark image, wherein the watermark image contains the target watermark information.

[0025] Based on the same inventive concept, the fifth aspect of the present disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the training method of the emotion determination model when executing the computer program.

[0026] Based on the same inventive concept, the sixth aspect of the present disclosure provides a non-transitory computer readable storage medium, which stores computer instructions for causing a computer to execute the training method of the emotion determination model.

[0027] It can be seen from the above that the present disclosure proposes a watermark image generation model training method, a watermark image generation method and equipment. The watermark image generation model includes a watermark coding and decoding module, a watermark feature mapping module and a first image information generation network, and the first image information generation network includes an adaptive attention module. The training data set is obtained, wherein the training data set includes original watermark information and an original image. The original image is input into a variational autoencoder, and the original image feature is output through the processing of the variational autoencoder. The original watermark information and the original image feature are input into the initial watermark coding and decoding module. The original watermark information and the original image feature are used to train the initial watermark coding and decoding module until the first target loss function corresponding to the initial watermark coding and decoding module converges, and the watermark coding and decoding module is obtained. First, the initial watermark coding and decoding module is trained to ensure the integrity and robustness of the watermark. The original watermark information is input into the watermark coding and decoding module to obtain the watermark feature. The original watermark information is input into the watermark feature mapping module to obtain the mapping feature. The watermark feature mapping module is used to map watermark features with different distributions to a unified feature space. The watermark feature and the mapping feature are input into the adaptive attention module in the initial first image information generation network, and the initial adaptive feature is obtained through the processing of the adaptive attention module. The initial adaptive feature is used to train the initial first image information generation network until the second target loss function corresponding to the initial first image information generation network converges, the first image information generation network training is completed, and the watermark image generation model is obtained. Through the fine-tuning training of the first image information generation network, the watermark image generation model obtained can better preserve the integrity and robustness of the watermark feature when generating an image. The image generated by the watermark image generation model can effectively maintain the recognition of the watermark under various conditions, thereby improving the overall watermark effect and meeting the flexibility of different watermarks. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the present disclosure or the related art, the following will briefly introduce the drawings needed to be used in the embodiments or related art descriptions. Obviously, the drawings in the following description are only embodiments of the present disclosure, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0029] Figure 1 The flowchart of the watermark image generation model training method of the embodiment of the present disclosure;

[0030] Figure 2 The flowchart of the watermark image generation method of the embodiment of the present disclosure;

[0031] Figure 3 The architecture diagram of the watermark image generation model training method of another embodiment of the present disclosure;

[0032] Figure 4 A model architecture schematic diagram of a watermark encoding module of another embodiment of the present disclosure;

[0033] Figure 5 A structural block diagram of a training device of a watermark image generation model of an embodiment of the present disclosure;

[0034] Figure 6 A structural block diagram of a watermark image generation device of an embodiment of the present disclosure;

[0035] Figure 7 A structural schematic diagram of an electronic device of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the present disclosure clearer, the present disclosure is further described in detail below with reference to specific embodiments and drawings.

[0037] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood as the general meaning understood by those skilled in the art to which the present disclosure belongs. The terms "first", "second", and similar terms used in the embodiments of the present disclosure do not represent any order, number, or importance, but are only used to distinguish different components. The terms "include", "contain", and similar terms mean that the elements or objects before the terms cover the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "up", "down", "left", "right", and the like only represent relative positional relationships, and when the absolute positions of the described objects change, the relative positional relationships may also change accordingly.

[0038] The terms involved in the present disclosure are explained as follows:

[0039] U-Net: image information generation network.

[0040] VAE: Variational Autoencoder (VAE), which describes observations in the latent space in a probabilistic way and has shown great application value in data generation.

[0041] WDM: Watermark Decoding Module (WDM).

[0042] WEM: Watermark Embedding Module (WEM).

[0043] Digital watermarking is a widely used copyright protection scheme in images, audio and video, and deep models. Traditional watermarking schemes achieve image copyright protection by using frequency domain transformation after image generation or by training watermark encoding and decoding networks. This method of embedding watermarks after generating images is called post-embedding watermarking. However, post-watermarking is vulnerable to watermark attacks that remove watermarks from images, such as image cutting, intelligent removal, and other technical means that cause the watermark in the image to be removed.

[0044] Therefore, recent research focuses on embedding watermarks imperceptibly in the diffusion process of the diffusion model, and the images generated based on this method carry watermark information. This method is called diffusion process watermarking. Among them, Stable Signature proposes a method using a pre-trained decoder. Specifically, the watermark is embedded in the variational autoencoder decoder of the diffusion model. However, this scheme requires training different VAE decoders for each different watermark, which makes it difficult for Stable Signature to meet the needs of thousands of users. In addition, malicious users can easily evade watermark verification by replacing the VAE decoder that does not embed the watermark.

[0045] In addition, it is proposed to hide the watermark in the frequency domain of the initial noise vector of the diffusion model, and then recover the noise by reversing the generated image to detect the watermark information. Although Tree-Rings does not require training, the detection result is largely dependent on the recovery process, and there are problems in identifying between multiple users. In summary, existing schemes cannot effectively solve the problem of replacing the VAE decoder to evade watermarking, and existing research focuses on embedding fixed watermark information in the diffusion process, which cannot meet the user's demand for embedding diversified watermarks.

[0046] Based on the above description, the present embodiment proposes a training method of a watermark image generation model, which includes a watermark encoding and decoding module, a watermark feature mapping module, and a first image information generation network, as shown in Figure 1 The method comprises the following steps:

[0047] Step 101, obtaining a training data set, wherein the training data set comprises original watermark information and an original image.

[0048] Step 102, inputting the original image into a variational autoencoder, processing the original image through the variational autoencoder, and outputting original image features.

[0049] Step 103, input the original watermark information and the original image features into the initial watermark encoding and decoding module, train the initial watermark encoding and decoding module using the original watermark information and the original image features until the first target loss function corresponding to the initial watermark encoding and decoding module converges, and obtain the watermark encoding and decoding module.

[0050] Step 104, input the original watermark information into the watermark encoding and decoding module to obtain watermark features.

[0051] Step 105, input the original watermark information into the watermark feature mapping module to obtain mapping features.

[0052] Step 106, input the watermark features and the mapping features into the adaptive attention module in the initial first image information generation network, process through the adaptive attention module to obtain initial adaptive features, train the initial first image information generation network using the initial adaptive features until the second target loss function corresponding to the initial first image information generation network converges, the first image information generation network training is completed, and a watermark image generation model is obtained.

[0053] In a specific implementation, a training data set is obtained, wherein the training data set includes original watermark information and original images, the original images are images without watermarks, and the original watermark information is binary watermark information.

[0054] The watermark encoding and decoding module includes a variational autoencoder, and in this embodiment, the variational autoencoder and the variational autoencoder are trained encoders and decoders. In this embodiment, the model parameters of the variational autoencoder and the variational autoencoder are frozen, that is, the variational autoencoder and the variational autoencoder are not trained, and only the watermark encoding module and the watermark decoding module are trained.

[0055] The original image is input into the variational autoencoder, processed through the variational autoencoder, and the original image features are output.

[0056] After the watermark encoding and decoding pre-training stage, the watermark encoding and decoding module is trained, and the specific process is as follows: input the original watermark information and the original image features into the initial watermark encoding and decoding module, train the initial watermark encoding and decoding module using the original watermark information and the original image features until the first target loss function corresponding to the initial watermark encoding and decoding module converges, and obtain the watermark encoding and decoding module.

[0057] After determining that the watermark encoding and decoding module is trained, the U-Net fine-tuning stage is performed, and the model parameters obtained by freezing the watermark encoding and decoding module. The original watermark information is input into the watermark encoding and decoding module to obtain watermark features.

[0058] The original watermark information is input into the watermark feature mapping module to obtain mapping features. The specific method of using the watermark feature mapping module to obtain mapping features is as follows:

[0059] The original watermark information is a watermark sequence, specifically a binary watermark information. The watermark sequence is converted into a vector of length r. bits, using embedding vectors and 0 to represent the binary states 0 and 1, where Initialized from a standard Gaussian distribution.

[0060] The calculation process of the watermark feature mapping module is as follows:

[0061]

[0062]

[0063] in Indicates the The sequence obtained after bit watermark mapping, Indicates the bit watermark sequence value, is the watermark sequence length. represents the watermark mapping feature matrix, (.) represents a function for creating a diagonal matrix. Finally, the diagonal matrix of the mapped watermark sequence is obtained as the watermark mapping feature, that is, the mapping feature.

[0064] The watermark features and the mapping features are input into an adaptive attention module in an initial first image information generation network, where the first image information generation network is a U-Net network. After being processed by the adaptive attention module, initial adaptive features are obtained. The initial first image information generation network is trained using the initial adaptive features until the second objective loss function corresponding to the initial first image information generation network converges, completing the training of the first image information generation network and obtaining a watermark image generation model.

[0065] The training data set is obtained through the above scheme, wherein the training data set includes original watermark information and an original image. The original image is input into the variational autoencoder, and the original image feature is output through the processing of the variational autoencoder. The original watermark information and the original image feature are input into the initial watermark encoding and decoding module. The initial watermark encoding and decoding module is trained by using the original watermark information and the original image feature until the first target loss function corresponding to the initial watermark encoding and decoding module converges, and the watermark encoding and decoding module is obtained. First, the initial watermark encoding and decoding module is trained to ensure the integrity and robustness of the watermark. The original watermark information is input into the watermark encoding and decoding module to obtain the watermark feature. The original watermark information is input into the watermark feature mapping module to obtain the mapping feature. The watermark feature mapping module is used to map watermark features of different distributions to a unified feature space. The watermark feature and the mapping feature are input into the adaptive attention module in the initial first image information generation network. The initial adaptive feature is obtained through the processing of the adaptive attention module. The initial first image information generation network is trained by using the initial adaptive feature until the second target loss function corresponding to the initial first image information generation network converges. The first image information generation network is trained to obtain the watermark image generation model. Through the fine-tuning training process of the first image information generation network, the watermark image generation model obtained can better preserve the integrity and robustness of the watermark feature when generating an image. The image generated by the watermark image generation model can effectively maintain the recognition of the watermark under various conditions, thereby improving the overall watermark effect and meeting the flexibility of different watermarks.

[0066] In some embodiments, step 103 specifically includes:

[0067] Step 1021, inputting the original watermark information into the initial watermark encoding module to output the initial watermark feature through the processing of the initial watermark encoding module;

[0068] Step 1022, performing fusion processing on the original image feature and the initial watermark feature to obtain an embedded image feature;

[0069] Step 1023, inputting the original image feature and the embedded image feature into the variational auto-decoder to output the initial watermark image through the processing of the variational auto-decoder, and determining a first loss function corresponding to the variational auto-decoder;

[0070] Step 1024, inputting the initial watermark image into the initial watermark decoding module, and decoding the initial watermark image by using the initial watermark decoding module to obtain training watermark information;

[0071] Step 1025, determining a second loss function corresponding to the initial watermark encoding module and the initial watermark decoding module according to the original watermark information and the training watermark information;

[0072] Step 1026, the first loss function and the second loss function are added to obtain a first target loss function corresponding to the initial watermark encoding and decoding module;

[0073] Step 1027, the initial watermark encoding module and the initial watermark decoding module are trained using the original watermark information and the original image feature until the first target loss function converges, and a watermark encoding and decoding module is obtained.

[0074] In specific implementation, the watermark encoding and decoding module includes a variational autoencoder, a watermark encoding module, a watermark decoding module, and a variational autoencoder. In this embodiment, the variational autoencoder and the variational autoencoder are both trained encoders and decoders. In this embodiment, the model parameters of the variational autoencoder and the variational autoencoder are frozen, that is, the variational autoencoder and the variational autoencoder are not trained, and only the watermark encoding module and the watermark decoding module are trained.

[0075] The original watermark information is input into the initial watermark encoding module, processed by the initial watermark encoding module, and the initial watermark feature is output. The function of the initial watermark encoding module is to convert the binary watermark information into a form suitable for embedding in the image feature, that is, to convert it into a two-dimensional feature map with the same size as the original image. The two-dimensional feature map is the initial watermark feature.

[0076] The original image feature and the initial watermark feature are fused to obtain an embedded image feature, wherein the fusion processing is matrix addition processing.

[0077] The original image feature and the embedded image feature are input into the variational autoencoder, and the original image feature and the embedded image feature are decoded by the variational autoencoder. The initial watermark image is output, and the first loss function corresponding to the variational autoencoder is determined. The initial watermark image contains the original watermark information and the image content contained in the original image.

[0078] The initial watermark image is input into the initial watermark decoding module, and the initial watermark decoding module is used to decode the initial watermark image to obtain training watermark information. The original watermark information and the training watermark information are compared to determine the second loss function corresponding to the initial watermark encoding module and the initial watermark decoding module.

[0079] In this embodiment, the second loss function is preferably a binary cross entropy loss (BCE Loss). The second loss function is used to evaluate the accuracy of watermark decoding.

[0080] The first loss function and the second loss function are added to obtain a first target loss function corresponding to the initial watermark encoding and decoding module. The initial watermark encoding module and the initial watermark decoding module are trained using the original watermark information and the original image features until the first target loss function converges, and the watermark encoding and decoding module is obtained.

[0081] Through the above scheme, the original watermark information is input into the initial watermark encoding module, processed by the initial watermark encoding module, and the initial watermark feature is output to ensure that the watermark can be successfully embedded into the original image feature. At the same time, the original watermark information and the training watermark information are compared to ensure the integrity and robustness of the watermark.

[0082] In some embodiments, the initial watermark encoding module includes an input layer, a linear layer, an activation layer, a convolution layer, and an output layer, and step 1021 specifically includes:

[0083] Step 10211, input the original watermark information into the initial watermark encoding module through the input layer;

[0084] Step 10212, converting the original watermark information using the linear layer to obtain linear conversion features;

[0085] Step 10213, the linear conversion features are non-linearly enhanced by the activation function of the activation layer to obtain non-linear enhanced features;

[0086] Step 10214, the non-linear enhanced features are adjusted in channel number using the convolution layer to obtain initial watermark features;

[0087] Step 10215, the initial watermark features are output from the initial watermark encoding module through the output layer.

[0088] In specific implementation, the initial watermark encoding module includes an input layer, a linear layer, an activation layer, a convolution layer, and an output layer, and the original watermark information is input into the initial watermark encoding module using the input layer.

[0089] The original watermark information is converted using the linear layer to obtain linear conversion features. Specifically, the linear layer converts one-dimensional watermark information with a length of 48 bits into one-dimensional information with a length of 64*64, and then uses Reshape to convert the one-dimensional information into linear conversion features.

[0090] The linear conversion features are non-linearly enhanced by the activation function of the activation layer, i.e., the non-linear representation of the linear conversion features is enhanced by the activation function, to obtain non-linear enhanced features. In this embodiment, the activation function is a SiLU activation function.

[0091] The convolutional layer is used to adjust the channel number of the nonlinear enhanced feature to obtain an initial watermark feature. In this embodiment, the convolutional layer only changes the channel number of the feature, without changing the size of the feature, and is used to enhance the ability of the model to extract the watermark feature. The initial watermark feature is output from the initial watermark encoding module through an output layer.

[0092] In this embodiment, the watermark decoding module is retrained using the watermark decoder model structure of StegaStamp to achieve the purpose of decoding the watermark information.

[0093] In some embodiments, step 1023 specifically includes:

[0094] Step 10231, inputting the original image feature into the variational autoencoder to obtain a first target original image through processing by the variational autoencoder;

[0095] Step 10232, inputting the embedded image feature into the variational autoencoder to obtain an initial watermark image and a second target original image through processing by the variational autoencoder;

[0096] Step 10233, determining a first loss function corresponding to the variational autoencoder according to the first target original image and the second target original image.

[0097] In specific implementation, the original image feature is input into the variational autoencoder to obtain the first target original image through processing by the variational autoencoder. The embedded image feature is input into the variational autoencoder to obtain the initial watermark image and the second target original image through processing by the variational autoencoder.

[0098] The first target original image and the second target original image are compared to determine the first loss function corresponding to the variational autoencoder. The first loss function is used to enhance the image quality by perceiving the distance between the features and to ensure the similarity between the image generated by the model and the original image.

[0099] The first loss function is a loss function determined according to a learned perceptual image patch similarity (LPIPS) and a peak regional variation loss (PRVL loss).

[0100] In this embodiment, the first loss function is expressed by a formula as follows:

[0101]

[0102] wherein, is the first loss function, is the learned perceptual image patch similarity, is a peak region change loss.

[0103] Further, the first target loss function corresponding to the initial watermark encoding and decoding module is expressed by a formula as:

[0104]

[0105] wherein, is the first target loss function, is a second loss function.

[0106] In some embodiments, the watermark encoding and decoding module further includes an image attack layer, and step 1024 specifically includes:

[0107] Step 10241, performing noise adding processing on the initial watermark image by using the image attack layer to obtain a target watermark image;

[0108] Step 10242, inputting the target watermark image into the initial watermark decoding module, and performing decoding processing on the target watermark image by using the initial watermark decoding module to obtain training watermark information.

[0109] In specific implementation, since various types of image attacks exist in actual scenes of image use, in the watermark encoding and decoding pre-training stage, the initial watermark image is processed by using the image attack layer before being input into the watermark decoding module. The specific manner is that a plurality of noise layers are introduced, and the noise layers include at least one of the following: JPEG compression, cropping and scaling, Gaussian blur, Gaussian noise and color jitter.

[0110] In the forward propagation process, the image attack layer randomly selects a noise layer from each noise layer according to a preset probability and applies the noise layer to the initial watermark image. That is, noise adding processing is performed on the initial watermark image to obtain a target watermark image. The target watermark image is input into the initial watermark decoding module, and decoding processing is performed on the target watermark image by using the initial watermark decoding module to obtain training watermark information.

[0111] Through the above scheme, the initial watermark image is processed by using the image attack layer to add noise, so that the disturbance of the input data has randomness each time the training is performed, thereby improving the robustness of the model.

[0112] In some embodiments, the watermark encoding and decoding module includes a variational autoencoder, a watermark encoding module, a watermark decoding module and a variational auto-decoder, and step 104 specifically includes:

[0113] Step 1041, inputting the original image into the variational autoencoder, and outputting original image features via the variational autoencoder processing;

[0114] Step 1042, input the original watermark information to the watermark encoding module, and obtain target watermark features through processing of the watermark encoding module.

[0115] Step 1043, fuse the original image features and the target watermark features to obtain watermark features.

[0116] In a specific implementation, the watermark encoding and decoding module includes a variational autoencoder, a watermark encoding module, a watermark decoding module, and a variational autoencoder. In this embodiment, it is the U-Net fine-tuning stage, in which the variational autoencoder, the watermark encoding module, the watermark decoding module, and the variational autoencoder in the watermark encoding and decoding module have been trained, and therefore the model parameters corresponding to the variational autoencoder, the watermark encoding module, the watermark decoding module, and the variational autoencoder are subjected to a freezing process. That is, in the U-Net fine-tuning stage, the variational autoencoder, the watermark encoding module, the watermark decoding module, and the variational autoencoder in the watermark encoding and decoding module are not trained.

[0117] In the U-Net fine-tuning stage, the previously generated watermark pattern is integrated and embedded into the U-Net. Specifically, the embedding of the watermark is completed using the LoRA technology. The original image is input to the variational autoencoder, and the original image features are output through processing of the variational autoencoder.

[0118] The original watermark information is input to the trained watermark encoding module, and target watermark features are obtained through processing of the watermark encoding module. The original image features and the target watermark features are fused to obtain watermark features, and the fusion processing is matrix addition processing.

[0119] Through the above scheme, the original image generates original image features through the variational autoencoder, which are combined with the target watermark features extracted by the watermark encoding module from the original watermark information to obtain watermark features. The watermark features contain information embedded in the watermark, forming a fused feature representation.

[0120] In some embodiments, the watermark image generation model further includes a second image information generation network, and the first image information generation network includes a denoising processing module and an adaptive attention module. Step 106 specifically includes:

[0121] Step 1061, input the watermark features to the denoising processing module, and perform denoising processing on the watermark features using the denoising processing module to obtain image segmentation features;

[0122] Step 1062, input the image segmentation features and the mapping features to the adaptive attention module, and obtain initial adaptive features through processing of the adaptive attention module;

[0123] Step 1063, input the initial adaptive feature into the denoising processing module, and use the denoising processing module to perform denoising processing on the initial adaptive feature to obtain an adaptive feature;

[0124] Step 1064, input the original image feature into the second image information generation network, and use the second image information generation network to perform denoising processing to obtain a target original image feature;

[0125] Step 1065, determine a second target loss function corresponding to the initial first image information generation network according to the adaptive feature and the target original image feature;

[0126] Step 1066, train the initial first image information generation network using the watermark feature and the mapping feature until the second target loss function corresponding to the initial first image information generation network converges, and the first image information generation network training is completed.

[0127] In specific implementation, the watermark image generation model further includes a second image information generation network, which is a trained U-Net network. In this embodiment, the network parameters of the second image information generation network are frozen, that is, the second image information generation network is not trained.

[0128] The watermark feature is input into the denoising processing module, and the denoising processing module is used to perform denoising processing on the watermark feature to obtain an image segmentation feature, which is the U-Net feature. The image segmentation feature and the mapping feature obtained by processing the watermark feature through the watermark feature mapping module are input into the adaptive attention module, and processed through the adaptive attention module to obtain an initial adaptive feature.

[0129] The initial adaptive feature is input into the denoising processing module, and the denoising processing module is used to perform denoising processing on the initial adaptive feature to obtain an adaptive feature. The original image feature is input into the second image information generation network, and the second image information generation network is used to perform denoising processing to obtain a target original image feature.

[0130] The adaptive feature and the target original image feature are compared to determine a second target loss function corresponding to the initial first image information generation network. In this embodiment, the second target loss function is a mean square error loss function, which is used to measure the pixel-level difference between the original U-Net generated feature and the LoRA fine-tuned feature. By minimizing the MSE Loss, the model can gradually adjust the generated feature representation, so that the embedded watermark information appears more accurately in the generated image, thereby improving the quality and robustness of the watermark.

[0131] In some embodiments, the adaptive attention module includes an input layer, a linear layer, a cross-attention layer, and an output layer, and step 1062 specifically includes:

[0132] Step 10621, input the image segmentation feature and the mapping feature into the adaptive attention module through the input layer;

[0133] Step 10622, dimension adjustment processing is performed on the image segmentation feature by using the linear layer to obtain a first dimension-reduced sequence feature;

[0134] Step 10623, dimension adjustment processing is performed on the mapping feature by using the linear layer to obtain a second dimension-reduced sequence feature;

[0135] Step 10624, weight adjustment is performed on the first dimension-reduced sequence feature and the second dimension-reduced sequence feature by using the cross-attention layer to obtain an initial adaptive feature.

[0136] In specific implementation, the adaptive attention module includes an input layer, a linear layer, a cross-attention layer, and an output layer, and the image segmentation feature and the mapping feature are input into the adaptive attention module through the input layer.

[0137] The dimension adjustment processing is performed on the image segmentation feature by using the linear layer to obtain a first dimension-reduced sequence feature, and the dimension adjustment processing is performed on the mapping feature by using the linear layer to obtain a second dimension-reduced sequence feature. The weight adjustment is performed on the first dimension-reduced sequence feature and the second dimension-reduced sequence feature by using the cross-attention layer to obtain an initial adaptive feature.

[0138] Through the above scheme, the adaptive attention module is used to first obtain dimension-reduced sequence features from the image segmentation feature and the mapping feature through the linear layer, and then calculate the correlation between the features in the cross-attention to generate a weighted feature representation, which is the initial adaptive feature. The adaptive process modifies the down-sampling layer of the U-Net, and this process allows the U-Net to more accurately embed the watermark feature into the target image during generation and achieve flexibility of the algorithm in processing different watermarks. The adaptive attention mechanism provides an effective solution to the single watermark embedding problem. According to different input watermarks, the attention of the U-Net model on different watermark features is dynamically adjusted to ensure that the model can better preserve the integrity and robustness of the watermark feature when generating an image. The generated image can effectively maintain the recognizability of the watermark under various conditions, thereby improving the overall watermark effect and meeting the flexibility of different watermarks.

[0139] Another embodiment of the present disclosure provides a watermark image generation method, which applies the watermark image generation model obtained by the above-mentioned embodiment, as shown in Figure 2 The method includes:

[0140] Step 201, obtaining an image to be processed and target watermark information;

[0141] Step 202, inputting the image to be processed and the target watermark information into a watermark image generation model, processing via the watermark image generation model to obtain a watermark image, wherein the watermark image contains the target watermark information.

[0142] In a specific implementation, the image to be processed and the target watermark information are obtained, and the image to be processed and the target watermark information are input into the trained watermark image generation model. Via processing of the watermark image generation model, a watermark image is obtained, wherein the watermark image contains the target watermark information.

[0143] Based on the same inventive concept, another embodiment of the present disclosure provides a training method of a watermark image generation model. The architecture diagram of the method is shown in Figure 3 The method specifically includes:

[0144] The training method of the watermark image generation model is divided into two stages: a watermark encoding and decoding pre-training stage and a U-Net fine-tuning stage. In the watermark encoding and decoding pre-training stage, a set of watermark encoding and decoding is trained by freezing all parameters of the diffusion model without interfering with the original ability of the model to ensure the effectiveness of the watermark. In the U-Net fine-tuning stage, the parameters of the watermark encoding and decoding in the first stage are frozen, and the blue snowflakes in the diagram represent the frozen model parameters. The LoRA method is used to effectively fine-tune the U-Net denoising part of the diffusion model to minimize the impact on the image generation ability and enable the U-Net to learn the watermark pattern in the first stage. At the same time, an adaptive attention mechanism is introduced in the U-Net fine-tuning stage, so that the model can be adjusted according to different watermark features, improving the flexibility and adaptability of the model.

[0145] In the watermark encoding and decoding pre-training stage, the function of the watermark encoding module (Watermark Embedding Module, WEM) is to convert the binary watermark information M (such as "101011...") into a two-dimensional feature map of the size of the image suitable for embedding into the image feature, so as to ensure that the watermark can be successfully embedded into the image feature. The fusion of the image feature and the watermark feature is performed by simple matrix addition. Then, the VAE decoder further decodes these features and generates an image carrying the watermark. The output watermark image is compared with the input watermark information by the watermark decoding module (Watermark Decoding Module, WDM) to ensure the integrity and robustness of the watermark.

[0146] The model architecture of the watermark encoding module WEM is as shown in Figure 4As shown, it is composed of linear layers, convolutional layers and nonlinear activation layers. Specifically, it contains one linear layer, four nonlinear layers and three convolutional layers. The linear layer converts one-dimensional watermark information of length 48 bits into one-dimensional information of length 64*64. Then, the one-dimensional information is converted into two-dimensional features by Reshape. The SiLU activation function is used to enhance the nonlinear representation of the features. Finally, the watermark feature map is obtained through the convolutional layer, and then fused with the image features. Among them, the convolutional module only changes the channel number of the feature map without changing the size of the feature map, which is used to enhance the ability of the model to extract watermark features.

[0147] Regarding the watermark decoding module WDM, the StegaStamp watermark decoder model structure is retrained to achieve the purpose of decoding the watermark information.

[0148] Since various types of image attacks exist in the actual scene of image use, in the watermark encoding and decoding pre-training stage, the watermark image is processed by the attack layer before being input to the watermark decoder WDM. Specifically, multiple noise layers are introduced, including JPEG compression, cropping and scaling, Gaussian blur, Gaussian noise and color jitter. During the forward propagation process, the image attack layer randomly selects one from each noise layer to apply to the input image according to the preset probability. By design, the disturbance of the input data is random each time the model is trained, thereby improving the robustness of the model.

[0149] In the watermark encoding and decoding pre-training stage, the loss function is represented by the formula:

[0150]

[0151] Among them, is the loss function, is the image perceptual similarity loss, is the peak regional variation loss, is the binary cross-entropy loss.

[0152] In this embodiment, the binary cross-entropy loss (BCE Loss) is used to evaluate the accuracy of watermark decoding, and the distance between the learned perceptual image patch similarity (LPIPS) and the peak regional variation loss (PRVL Loss) is used to enhance the image quality, ensuring the similarity between the image generated by the model and the original image.

[0153] In the U-Net fine-tuning stage, the goal is to integrate the previously generated watermark pattern set into the U-Net. To achieve this goal, the LoRA technique is used to complete the embedding of the watermark. Overall, the original image is encoded by the VAE encoder to generate features, which are merged with the watermark features extracted by the watermark encoder WEM. At this time, the features contain the information of the watermark embedding, forming a fused feature representation. To further enhance the expression and embedding of the watermark features, an adaptive attention mechanism is introduced, which allows the model to adaptively process the watermark mapping features and the U-Net denoising features. The adaptive attention mechanism provides an effective solution to the single watermark embedding problem. Specifically, according to the different input watermarks, the attention of the U-Net model on different watermark features is dynamically adjusted, which ensures that the model can better preserve the integrity and robustness of the watermark features when generating images. The generated images effectively maintain the recognizability of the watermark under various conditions, thereby improving the overall watermark effect and meeting the flexibility of different watermarks.

[0154] The adaptive process enables the model to handle the distribution differences between different watermark features by introducing a flexible adaptive attention mechanism, without the need to retrain the model for each new watermark. When faced with diverse watermark features, the model can still maintain high embedding and decoding accuracy, thereby solving the identification problem between multiple users.

[0155] The adaptive process includes a watermark feature mapping module and an adaptive attention mechanism. The watermark feature mapping module is a crucial part of the adaptive process, which is used to map different distributed watermark features to a unified feature space. Specifically, the watermark sequence of length is converted to a vector of length r. For the th bit of the watermark, the embedding vector and 0 are used to represent binary states 0 and 1, where is initialized by a standard Gaussian distribution. The calculation process of the watermark feature mapping module is as follows:

[0156]

[0157]

[0158] where represents the sequence obtained after mapping the th bit of the watermark, represents the value of the th bit of the watermark sequence, is the length of the watermark sequence. represents the watermark mapping feature matrix, (.) represents a function for creating a diagonal matrix. Finally, the diagonal matrix of the mapped watermark sequence is obtained as the watermark mapping feature.

[0159] As Figure 3 shown, the adaptive attention mechanism is realized through the cross-attention module, which further adjusts the weight of the features in the generation process by combining the features extracted by the U-Net and the watermark features. The module first obtains the dimension-reduced sequence features from the features from the U-Net and the watermark features through a linear layer, and then calculates the correlation between the features in the cross-attention to generate a weighted feature representation. Finally, the adaptive features are input to the next U-Net network. Specifically, the adaptive process modifies the down-sampling layer of the U-Net. This process allows the U-Net to more accurately embed the watermark features into the target image during the generation process and achieve flexibility of the algorithm in processing different watermarks.

[0160] In the U-Net fine-tuning stage, the mean square error loss function (MSE Loss) is used in this embodiment to measure the pixel-level difference between the original U-Net generated features and the LoRA fine-tuned features. By minimizing the MSE Loss, the model can gradually adjust the generated feature representation so that the embedded watermark information appears more accurately in the generated image, thereby improving the quality and robustness of the watermark.

[0161] It should be noted that the method of the embodiments of the present disclosure can be executed by a single device, such as a computer or a server, etc. The method of the embodiments can also be applied to a distributed scenario, which is completed by multiple devices cooperating with each other. In this distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiments of the present disclosure, and the multiple devices can interact with each other to complete the method.

[0162] It should be noted that some embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps in a claim can be performed in an order different than the order described above and still achieve a desired result. Additionally, the process depicted in the figures can not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0163] Based on the same inventive concept, the present disclosure also provides a watermark image generation model training device corresponding to any of the above-mentioned embodiment methods, which includes a watermark encoding and decoding module, a watermark feature mapping module, and a first image information generation network, comprising:

[0164] Reference Figure 5 , Figure 5 The watermark image generation model training device of the embodiments includes:

[0165] The data acquisition module 501 is configured to acquire a training data set, wherein the training data set comprises original watermark information and an original image.

[0166] The original image feature determination module 502 is configured to input the original image into a variational autoencoder, process via the variational autoencoder, and output an original image feature.

[0167] The first training module 503 is configured to input the original watermark information and the original image feature into an initial watermark encoding and decoding module, train the initial watermark encoding and decoding module by using the original watermark information and the original image feature until a first target loss function corresponding to the initial watermark encoding and decoding module converges, and obtain a watermark encoding and decoding module.

[0168] The watermark feature determination module 504 is configured to input the original watermark information into the watermark encoding and decoding module to obtain a watermark feature.

[0169] The mapping feature determination module 505 is configured to input the original watermark information into a watermark feature mapping module to obtain a mapping feature.

[0170] The second training module 506 is configured to input the watermark feature and the mapping feature into an adaptive attention module in an initial first image information generation network, process via the adaptive attention module to obtain an initial adaptive feature, train the initial first image information generation network by using the initial adaptive feature until a second target loss function corresponding to the initial first image information generation network converges, and train the first image information generation network to obtain a watermark image generation model.

[0171] In some embodiments, the first training module 503 specifically comprises:

[0172] The initial watermark feature determination unit is configured to input the original watermark information into an initial watermark encoding module, process via the initial watermark encoding module, and output an initial watermark feature.

[0173] The fusion unit is configured to fuse and process the original image feature and the initial watermark feature to obtain an embedded image feature.

[0174] The first loss function determination unit is configured to input the original image feature and the embedded image feature into a variational auto-decoder, process via the variational auto-decoder, output an initial watermark image, and determine a first loss function corresponding to the variational auto-decoder.

[0175] The decoding processing unit is configured to input the initial watermark image into an initial watermark decoding module, perform decoding processing on the initial watermark image by using the initial watermark decoding module, and obtain training watermark information.

[0176] The second loss function determination unit is configured to determine a second loss function corresponding to the initial watermark encoding module and the initial watermark decoding module according to the original watermark information and the training watermark information.

[0177] The first target loss function determination unit is configured to add the first loss function and the second loss function to obtain a first target loss function corresponding to the initial watermark encoding and decoding module.

[0178] The training unit is configured to train the initial watermark encoding module and the initial watermark decoding module by using the original watermark information and the original image feature until the first target loss function converges, so as to obtain the watermark encoding and decoding module.

[0179] In some embodiments, the initial watermark encoding module includes an input layer, a linear layer, an activation layer, a convolution layer, and an output layer, and the initial watermark feature determination unit is specifically configured to:

[0180] input the original watermark information into the initial watermark encoding module through the input layer;

[0181] convert the original watermark information by using the linear layer to obtain linear conversion features;

[0182] nonlinearly enhance the linear conversion features by using an activation function of the activation layer to obtain nonlinear enhancement features;

[0183] adjust the channel number of the nonlinear enhancement features by using the convolution layer to obtain the initial watermark features;

[0184] output the initial watermark features from the initial watermark encoding module through the output layer.

[0185] In some embodiments, the first loss function determination unit is specifically configured to:

[0186] input the original image feature into the variational autoencoder to obtain a first target original image through the variational autoencoder processing;

[0187] input the embedded image feature into the variational autoencoder to obtain an initial watermark image and a second target original image through the variational autoencoder processing;

[0188] determine a first loss function corresponding to the variational autoencoder according to the first target original image and the second target original image.

[0189] In some embodiments, the watermark encoding and decoding module further includes an image attack layer, and the decoding processing unit is specifically configured to:

[0190] perform noise adding processing on the initial watermark image by using the image attack layer to obtain a target watermark image;

[0191] The target watermark image is input into the initial watermark decoding module, and the target watermark image is decoded by using the initial watermark decoding module to obtain training watermark information.

[0192] In some embodiments, the watermark coding module includes a variational autoencoder, a watermark encoding module, a watermark decoding module, and a variational autoencoder, and the watermark feature determination module 504 is specifically configured to:

[0193] The original image is input into the variational autoencoder, and the original image features are output through the processing of the variational autoencoder;

[0194] The original watermark information is input into the watermark encoding module, and the target watermark features are obtained through the processing of the watermark encoding module;

[0195] The original image features and the target watermark features are fused to obtain watermark features.

[0196] In some embodiments, the watermark image generation model further includes a second image information generation network, the first image information generation network includes a denoising processing module and an adaptive attention module, and the second training module 506 specifically includes:

[0197] The image segmentation feature determination unit is configured to input the watermark features into the denoising processing module, and to obtain image segmentation features by using the denoising processing module to process the watermark features;

[0198] The initial adaptive feature determination unit is configured to input the image segmentation features and the mapping features into the adaptive attention module, and to obtain initial adaptive features by using the adaptive attention module to process the image segmentation features and the mapping features;

[0199] The adaptive feature determination unit is configured to input the initial adaptive features into the denoising processing module, and to obtain adaptive features by using the denoising processing module to process the initial adaptive features;

[0200] The target original image feature determination unit is configured to input the original image features into the second image information generation network, and to obtain target original image features by using the second image information generation network to process the original image features;

[0201] The second target loss function determination unit is configured to determine a second target loss function corresponding to the initial first image information generation network according to the adaptive features and the target original image features;

[0202] The training unit is configured to train the initial first image information generation network by using the watermark features and the mapping features until the second target loss function corresponding to the initial first image information generation network converges, and the first image information generation network training is completed.

[0203] In some embodiments, the adaptive attention module comprises an input layer, a linear layer, a cross-attention layer, and an output layer, the initial adaptive feature determination unit is specifically configured to:

[0204] The image segmentation feature and the mapping feature are input to the adaptive attention module through the input layer;

[0205] The linear layer is used to perform dimension adjustment processing on the image segmentation feature to obtain a first dimension-reduced sequence feature;

[0206] The linear layer is used to perform dimension adjustment processing on the mapping feature to obtain a second dimension-reduced sequence feature;

[0207] The cross-attention layer is used to perform weight adjustment on the first dimension-reduced sequence feature and the second dimension-reduced sequence feature to obtain the initial adaptive feature.

[0208] Based on the same inventive concept, the present disclosure also provides a watermark image generation device corresponding to any of the above-mentioned embodiment methods, as shown in Figure 6 The device comprises:

[0209] A data acquisition module 601 configured to acquire a to-be-processed image and target watermark information;

[0210] A watermark image generation module 602 configured to input the to-be-processed image and the target watermark information into a watermark image generation model, and obtain a watermark image through processing of the watermark image generation model, wherein the watermark image contains the target watermark information.

[0211] For the sake of description, the above device is described in various modules according to functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0212] The device of the above-mentioned embodiment is used to implement the training method of the corresponding emotion determination model in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described here.

[0213] Based on the same inventive concept, the present disclosure also provides an electronic device corresponding to any of the above-mentioned embodiment methods, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any one of the above-mentioned embodiments when executing the program.

[0214] Figure 7A more specific electronic device hardware structure schematic diagram provided by the embodiment is shown. The device can include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 for internal communication.

[0215] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present specification.

[0216] The memory 1020 can be implemented by a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0217] The input / output interface 1030 is configured to connect an input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0218] The communication interface 1040 is configured to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or through a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0219] The bus 1050 includes a channel to transmit information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0220] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain components necessary to implement the embodiments of the present application, and does not necessarily contain all the components shown in the figure.

[0221] The electronic device of the above embodiment is used to implement the corresponding method in any of the preceding embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0222] Based on the same inventive concept, the disclosure also provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to execute the method of any of the above embodiments.

[0223] The computer readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be realized by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0224] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute the method of any of the above embodiments, and have the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0225] It can be understood that before using the technical solutions of various embodiments in the present disclosure, the user will be informed of the type, use range, use scenario, etc. of the personal information involved in a proper manner, and the authorization of the user will be obtained.

[0226] For example, in response to receiving an active request of a user, a prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using personal information of the user. Thus, the user can autonomously select whether to provide the personal information to the software or hardware such as an electronic device, an application program, a server or a storage medium performing the operation of the technical solution of the present disclosure according to the prompt information.

[0227] As an optional but non-limiting implementation, in response to receiving an active request of a user, the prompt information can be sent to the user in the form of a pop-up window, and the prompt information can be presented in the form of text in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0228] It can be understood that the above notification and obtaining of user authorization process is only illustrative, and does not limit the implementation of the present disclosure, and other ways that meet the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0229] Those skilled in the art will understand that the discussion of any of the above embodiments is merely exemplary and is not intended to suggest that the scope of the present disclosure (including claims) is limited to these examples; the above embodiments or technical features among different embodiments can also be combined, steps can be implemented in any order, and there are many other variations of the aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of brevity. Therefore, the above description is not intended to limit the scope of the present disclosure (including claims).

[0230] In addition, in order to simplify the description and discussion, and so as not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections of integrated circuit (IC) chips and other components can or can not be shown in the provided drawings. In addition, the devices can be shown in the form of block diagrams in order to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented to implement the embodiments of the present disclosure (i.e., these details should be fully within the understanding of those skilled in the art). Where specific details (e.g., circuitry) are set forth in order to describe an illustrative embodiment of the present disclosure, it will be apparent to those skilled in the art that the present disclosure can be practiced without these specific details or with variations on these specific details. Therefore, these descriptions should be considered as illustrative rather than limiting.

[0231] While the present disclosure has been described in connection with certain embodiments thereof, many modifications, substitutions, and variations will be apparent to those of ordinary skill in the art from the foregoing description. For instance, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0232] Embodiments of the present disclosure are intended to cover all such alternatives, modifications, and variations as falling within the broad scope of the appended claims. Accordingly, any one or more of the above-described embodiments can be combined with any one or more of the other embodiments, and the application should not be construed as limited to only those embodiments described and shown in the attached drawings and following detailed description.

Claims

1. A method for training a watermark image generation model, characterized by, The watermark image generation model comprises a watermark coding and decoding module, a watermark feature mapping module and a first image information generation network, and the first image information generation network comprises an adaptive attention module; Comprise: Obtain a training data set, wherein the training data set comprises original watermark information and an original image; Input the original image into a variational autoencoder, process the original image via the variational autoencoder, and output original image features; Input the original watermark information and the original image features into an initial watermark coding and decoding module, train the initial watermark coding and decoding module using the original watermark information and the original image features until a first target loss function corresponding to the initial watermark coding and decoding module converges, and obtain a watermark coding and decoding module; Input the original watermark information into the watermark coding and decoding module to obtain watermark features; Input the original watermark information into the watermark feature mapping module to obtain mapping features; Input the watermark features and the mapping features into the adaptive attention module in the initial first image information generation network, process the watermark features and the mapping features via the adaptive attention module to obtain initial adaptive features, train the initial first image information generation network using the initial adaptive features until a second target loss function corresponding to the initial first image information generation network converges, the first image information generation network is trained, and a watermark image generation model is obtained; The inputting of the original watermark information and the original image features into the initial watermark coding and decoding module, the training of the initial watermark coding and decoding module using the original watermark information and the original image features until the first target loss function corresponding to the initial watermark coding and decoding module converges, and the obtaining of the watermark coding and decoding module comprise: Input the original watermark information into an initial watermark encoding module, process the original watermark information via the initial watermark encoding module, and output initial watermark features; Fuse the original image features and the initial watermark features to obtain embedded image features; Input the original image features and the embedded image features into a variational auto-decoder, process the original image features and the embedded image features via the variational auto-decoder, output an initial watermark image, and determine a first loss function corresponding to the variational auto-decoder; Input the initial watermark image into an initial watermark decoding module, decode the initial watermark image using the initial watermark decoding module to obtain training watermark information; Determine a second loss function corresponding to the initial watermark encoding module and the initial watermark decoding module according to the original watermark information and the training watermark information; Add the first loss function and the second loss function to obtain the first target loss function corresponding to the initial watermark coding and decoding module; Train the initial watermark encoding module and the initial watermark decoding module using the original watermark information and the original image features until the first target loss function converges, and obtain the watermark coding and decoding module.

2. The method of claim 1, wherein, The initial watermark encoding module comprises an input layer, a linear layer, an activation layer, a convolution layer and an output layer, The inputting of the original watermark information into the initial watermark encoding module and the processing of the original watermark information via the initial watermark encoding module to output the initial watermark features comprise: Input the original watermark information into the initial watermark encoding module through the input layer; Convert the original watermark information using the linear layer to obtain linear conversion features; The linearly converted features are subjected to nonlinear enhancement processing through an activation function of an activation layer to obtain nonlinearly enhanced features; The nonlinearly enhanced features are subjected to channel number adjustment through a convolution layer to obtain convolution features; The convolution features are output from the initial watermark encoding module through an output layer as initial watermark features.

3. The method of claim 1, wherein, The original image features and the embedded image features are input into the variational autoencoder, processed through the variational autoencoder, and output as an initial watermark image. A first loss function corresponding to the variational autoencoder is determined, including: The original image features are input into the variational autoencoder, processed through the variational autoencoder, and obtained as a first target original image. The embedded image features are input into the variational autoencoder, processed through the variational autoencoder, and obtained as an initial watermark image and a second target original image. The first loss function corresponding to the variational autoencoder is determined according to the first target original image and the second target original image.

4. The method of claim 1, wherein, The watermark encoding and decoding module further includes an image attack layer, The initial watermark image is input into the initial watermark decoding module, and the initial watermark decoding module is used to decode the initial watermark image to obtain training watermark information, including: The initial watermark image is subjected to noise addition processing through the image attack layer to obtain a target watermark image. The target watermark image is input into the initial watermark decoding module, and the initial watermark decoding module is used to decode the target watermark image to obtain training watermark information.

5. The method of claim 1, wherein, The watermark encoding and decoding module includes a variational autoencoder, a watermark encoding module, a watermark decoding module, and a variational autoencoder, The original watermark information is input into the watermark encoding and decoding module to obtain watermark features, including: The original image is input into the variational autoencoder, processed through the variational autoencoder, and output as original image features. The original watermark information is input into the watermark encoding module, processed through the watermark encoding module, and obtained as target watermark features. The original image features and the target watermark features are fused to obtain watermark features.

6. The method of claim 5, wherein, The watermark image generation model further includes a second image information generation network, and the first image information generation network includes a denoising processing module and an adaptive attention module, The watermark features and the mapping features are input into the adaptive attention module in the initial first image information generation network, processed through the adaptive attention module, and obtained as initial adaptive features. The initial first image information generation network is trained using the initial adaptive features until a second target loss function corresponding to the initial first image information generation network converges, the first image information generation network training is completed, and a watermark image generation model is obtained, including: The watermark features are input into the denoising processing module, and the denoising processing module is used to denoise the watermark features to obtain image segmentation features. The image segmentation features and the mapping features are input into the adaptive attention module, processed through the adaptive attention module, and obtained as initial adaptive features. The initial adaptive features are input into the denoising processing module, and the denoising processing module is used to denoise the initial adaptive features to obtain adaptive features. The original image features are input into the second image information generation network, and the second image information generation network is used to denoise to obtain target original image features. The second target loss function corresponding to the initial first image information generation network is determined according to the adaptive feature and the target original image feature; The initial first image information generation network is trained by using the watermark feature and the mapping feature until the second target loss function corresponding to the initial first image information generation network converges, and the first image information generation network training is completed.

7. The method of claim 6, wherein, The adaptive attention module comprises an input layer, a linear layer, a cross-attention layer and an output layer, The image segmentation feature and the mapping feature are input into the adaptive attention module, and the initial adaptive feature is obtained through processing of the adaptive attention module, comprising: The image segmentation feature and the mapping feature are input into the adaptive attention module through the input layer; The image segmentation feature is processed by the linear layer for dimension adjustment to obtain a first dimension-reduced sequence feature; The mapping feature is processed by the linear layer for dimension adjustment to obtain a second dimension-reduced sequence feature; The first dimension-reduced sequence feature and the second dimension-reduced sequence feature are processed by the cross-attention layer for weight adjustment to obtain the initial adaptive feature.

8. A method of generating a watermark image, characterized by, The watermark image generation model obtained by any one of claims 1 to 7 is applied, comprising: Obtaining a to-be-processed image and target watermark information; The to-be-processed image and the target watermark information are input into the watermark image generation model, and the watermark image is obtained through processing of the watermark image generation model, wherein the watermark image contains the target watermark information.

9. An electronic device, comprising: A computer program product comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the method of any one of claims 1 to 8.

Images