Hidden watermarking method for image generation model

Abstract

The invention discloses an image generation model-oriented implicit watermarking method, which comprises an implicit watermark embedding operation and an implicit watermark extraction operation, and is characterized in that during the implicit watermark embedding operation, watermark information is expanded and then encrypted according to binary array segments of original noise; the binary array of the original noise is adjusted through parity matching to embed the encrypted watermark data, an adjusted binary array of the original noise is obtained, and based on the adjusted binary array of the original noise, noise reduction and remodeling are carried out, and then image generation is carried out on the remodeled noise; during hidden watermark extraction, encrypted watermark data extraction is carried out by adopting logic similar to embedding operation, watermark data is obtained based on the extracted encrypted watermark data, then error correction is carried out, and watermark information is obtained; the method has the advantages that the watermark is effectively embedded, the quality of the generated image is ensured, the diversity of the generated image can be ensured, and negative effects on the output fidelity of the image generation model and the real representation of data are avoided.

Description

Technical Field

[0001] The present invention relates to a method for embedding invisible watermarks, and more particularly to a method for embedding invisible watermarks for an image generation model. Background Art

[0002] Image generation models are an important technology in the field of artificial intelligence and have received extensive attention and applications in recent years. By learning and modeling a large amount of image data, image generation models can generate image data with a very high degree of similarity to real image data. Based on the generated image data, an image decoder can generate images with a very high degree of similarity, which are widely used in fields such as image generation, image restoration, and virtual reality. Currently, there are mainly two common types of image generation models: image generation models based on generative adversarial networks (GANs) and image generation models based on diffusion models.

[0003] With the in-depth application of image generation models in industry and commerce, the protection of the originality and copyright tracking of images generated based on the generated image data have become key issues. To solve this problem, invisible watermark technology has gradually been introduced into image generation models. Invisible watermark technology is a means of embedding imperceptible information (i.e., watermark information) in an image, which does not affect the visual quality of the image but can effectively track the source of the image and prevent unauthorized use. Therefore, designing an effective invisible watermark method for an image generation model requires ensuring that the embedded watermark does not affect the image generation quality and maintaining high efficiency in tracking and copyright protection.

[0004] Currently, there are already some invisible watermark methods for image generation models. For example, the invisible watermark method based on Gaussian noise mapping embeds a unique watermark while generating an image by mapping the watermark into Gaussian noise. This method has a relatively small impact on the image quality when embedding the watermark and can ensure the image quality. However, this method embeds the watermark by restricting the sampling of the latent space to a certain fixed quadrant, but this fixed sampling method will significantly reduce the diversity of the generated images. This problem has a negative impact on the output fidelity and data true representation of the image generation model. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an invisible watermark method for an image generation model that can effectively embed a watermark, ensure the quality of the generated images, and at the same time ensure the diversity of the generated images without having a negative impact on the output fidelity and data true representation of the image generation model.

[0006] The technical solution adopted by the present invention to solve the above technical problems is as follows: A method for embedding invisible watermark in an image generation model, including an invisible watermark embedding operation and an invisible watermark extraction operation. The specific process of the invisible watermark embedding operation is as follows: First, expand the original noise in one dimension and perform binarization processing to obtain a binarized array of the original noise. The binarized array of the original noise is a one-dimensional array. Average-segment the binarized array of the original noise according to a fixed segment length, and record the number of segments after segmentation as Expand the watermark information in the form of binary data to obtain extended watermark information with the number of data bits equal to and encrypt the extended watermark information to obtain encrypted watermark data. The encrypted watermark data is a one-dimensional array, and its length is equal to Based on the encrypted watermark data, adjust the binarized array of the original noise through parity matching to embed the encrypted watermark data, obtain an adjusted binarized array of the original noise, restore the adjusted binarized array of the original noise to obtain a restored noise that conforms to the Gaussian distribution. The statistical distribution characteristics of the restored noise are the same as those of the original noise, and at the same time, the watermark information is retained. Then, reshape the restored noise according to the size of the original noise to obtain a reshaped noise. First, use an image generation model to generate reshaped noise data, and then use an image decoder to generate an image from the reshaped noise data, generating an image containing the encrypted watermark data. The specific process of the invisible watermark extraction is as follows: First, use an image encoder to extract the noise containing the encrypted watermark data from the image containing the encrypted watermark data, then expand the noise containing the encrypted watermark data in one dimension and perform binarization processing to obtain a binarized noise array containing the encrypted watermark data. The length of the binarized noise array containing the encrypted watermark data is equal to the length of the binarized array of the original noise. Extract the encrypted watermark data from the binarized noise array containing the encrypted watermark data, then decrypt the extracted encrypted watermark data to obtain the watermark data. Finally, correct the errors in the watermark data to obtain the watermark information.

[0007] Compared with the prior art, the advantages of the present invention are as follows: First, the binary array of the original noise is evenly segmented according to a fixed segment length. According to the number of segments after segmentation, the watermark information in the form of binary data is extended until the number of data bits is equal to the number of segments after segmentation, and then encrypted to obtain encrypted watermark data. Then, based on the encrypted watermark data, the binary array of the original noise is adjusted through parity matching to embed the encrypted watermark data, obtaining an adjusted binary array of the original noise. When the adjusted binary array of the original noise is restored to a restored noise conforming to the Gaussian distribution, the statistical distribution characteristics of the original noise are maintained. This process is achieved through mathematical transformation, ensuring that the restored noise input to the image generation model has no significant difference in characteristics from the original noise without watermark embedding. Therefore, the input received by the image generation model is the same as that in the conventional generation process, and the image generated by the final image decoder will not degrade in visual quality due to watermark embedding, ensuring that the quality of the generated image does not decrease. At the same time, when embedding the watermark (i.e., the encrypted watermark data), the method of restricting noise sampling is avoided. The binary array of the original noise is adjusted through parity matching to embed the encrypted watermark data, that is, only minor and scattered local adjustments are made to the binary array of the original noise, and they are distributed throughout the restored noise, without changing the overall sampling range or distribution characteristics of the original noise, and being statistically equivalent to the original noise. Therefore, the diversity of the restored noise is retained, and further the diversity of the generated images is ensured. Since the overall randomness and diversity of the original noise are not changed during the watermark embedding process, the output fidelity of the image generation model (i.e., the similarity between the generated image and the expected output of the image generation model) is maintained. At the same time, the generated image can still truly reflect the image data distribution generated by the original noise, without deviation or distortion caused by watermark embedding, ensuring the diversity and fidelity of the generated image. Thus, the present invention can effectively embed the watermark, ensure the quality of the generated image, and at the same time ensure the diversity of the generated image, without having a negative impact on the output fidelity of the image generation model and the true representation of the data.

[0008] Further, based on the encrypted watermark data, the specific process of adjusting the binary array of the original noise through parity matching to embed the encrypted watermark data to obtain an adjusted binary array of the original noise is as follows: Let the th segment data of the binary array of the original noise correspond to the th bit element of the encrypted watermark data. Respectively judge whether the sum of each segment data of the binary array of the original noise and the one-bit element corresponding to this segment data in the encrypted watermark data are both odd or both even. If so, the segment data of the binary array of the original noise remains unchanged. If not, the first bit element in the segment data of the binary array of the original noise is flipped to realize the adjustment of the binary array of the original noise, thereby obtaining an adjusted binary array of the original noise.

[0009] Further, the specific process of extracting the encrypted watermark data from the binarized noise array containing the encrypted watermark data is as follows: The binarized noise array containing the encrypted watermark data is evenly segmented, and the number of segments is also Set the encrypted watermark data to be extracted as a one-dimensional array with a length of Take the th segment of data in the binarized noise array containing the watermark data and correspond it to the th element of the encrypted watermark data to be extracted. If the sum of the th segment of data in the binarized array containing the watermark data is odd, then assign the value 1 to the th element of the encrypted watermark data to be extracted. If the sum of the th segment of data in the binarized noise array containing the watermark data is even, then assign the value 0 to the th element of the encrypted watermark data to be extracted. Thus, the encrypted watermark data with a length equal to and in the form of a one-dimensional array is obtained.

[0010] Further, the specific process of expanding the original noise in one dimension and performing binarization processing to obtain the binarized array of the original noise is as follows:

[0011] S1. Denote the original noise as Z, denote the number of channels contained in the original noise Z as c, and denote the size of each channel of the original noise Z as h×w, where h represents the length and w represents the width. Flatten the two-dimensional array of each channel of the original noise Z into a one-dimensional array in row-major order, and then splice the one-dimensional arrays of all channels in channel order to form a one-dimensional array with a length of c×h×w, denoted as Z f , denote the length of Z f as N, N = c×h×w, and define Z f [i] as the f ith element in Z

[0012] S2. Perform binarization conversion on each element of the one-dimensional array Z f to obtain the corresponding binarized data. Among them, the binarized data corresponding to the f ith element Z f [i] of the one-dimensional array Z z is denoted as B f [i], and the f ith element Z z [i] of the one-dimensional array Z

[0013]

[0014] where Φ(·) is the cumulative distribution function of the standard normal distribution;

[0015] S3. Use the one-dimensional array Z obtained in step S2 f The binarized data corresponding to each element of z forms the binarized array B of the original noise, where B z is a one-dimensional array with a length of N, and the i-th element of B z is B z [i].

[0016] Furthermore, the watermark information in binary data form is extended to obtain extended watermark information with the number of data bits equal to , and the specific process of encrypting the extended watermark information to obtain the encrypted watermark data is as follows:

[0017] A1. Set a positive integer l that is divisible by N w , and take every l z bits of the binarized array B of the original noise as a piece of data, so that the binarized array B of the original noise w is evenly divided into z segments of data, segments of data.

[0018] A2. Denote the watermark information in binary data form as M. M is a one-dimensional array, and denote the length of the watermark information M as k, where

[0019] A3. Judge whether is an integer. If so, repeat the watermark information M times to form the extended watermark information M r , M d = M, If not, repeat the watermark information M times and then fill zeros at the end to form the extended watermark information M r , M p = M, is bits, and each bit is 0, represents the ceiling symbol;

[0020] A4. Preset the key Key and use the ChaCha20 stream cipher algorithm to encrypt the extended watermark information M r to obtain the encrypted watermark data M with a length of enc。

[0021] Furthermore, the specific process of restoring the adjusted binary array of the original noise to obtain the restored noise conforming to the Gaussian distribution is as follows:

[0022] B1. Denote the adjusted binary array of the original noise as B z ′, and denote the i-th element of B z ′ as B z ′[i]. Denote the cumulative distribution function value of B z ′[i] as θ i , and calculate θ using formula (2): i :

[0023]

[0024] where u i represents a random variable obtained by independently sampling from a uniform distribution defined on the closed interval [0, 0.5];

[0025] B2. Use the probability distribution function of the standard normal distribution to map θ i to obtain the corresponding Gaussian distribution value, denoted as Z f ′[i], as shown in formula (3):

[0026] Z f ′[i] = Φ -1 (θ i ) (3)

[0027] where Φ -1 (·) is the probability distribution function;

[0028] B3. Use the Gaussian distribution values obtained in step B2 to form the restored one-dimensional Gaussian noise array Z f ′, and the i-th element of Z f ′ is Z f ′[i]; reshape the restored one-dimensional Gaussian noise array Z f ′ according to the size of the original noise Z to obtain the restored noise Z′ conforming to the Gaussian distribution.

[0029] Furthermore, the specific process of decrypting the extracted encrypted watermark data to obtain the watermark data and correcting the watermark data to obtain the watermark information is as follows

[0030] C1. Denote the extracted encrypted watermark data as M′ enc , and use the ChaCha20 stream cipher algorithm and the key Key to decrypt the extracted encrypted watermark data M e ′ nc to obtain the watermark data, and denote it as M r′, watermark data M r The length of ′ is

[0031] C2. Set the extended parameter ratio Determine whether Q is an integer. If so, execute the complete error correction process to obtain the watermark information M′; if Q is a non-integer, execute the error correction process including truncation to obtain the watermark information M′;

[0032] The complete segment error correction process includes the following steps:

[0033] Step C2.1: The watermark data M r Each k-bit element of ′ is divided into a piece of data, and Q pieces of data are obtained, and the length of each piece of data is k;

[0034] Step C2.2: Count the watermark data M respectively r The number of elements with the jth bit set to 1 in the Q segment data of ′, where j = 1, 2, ..., k. If the watermark data M r The number of elements with the jth bit set to 1 in the Q segment of ' is greater than or equal to Then let the j-th element of the watermark information M′ be 1, otherwise it is 0;

[0035] The error correction process including truncation includes the following steps:

[0036] Step C2.1: Set the maximum number of complete segments Indicates the floor symbol;

[0037] Step C2.2: Extract watermark data M′ r The first t*k elements of r , * is the multiplication symbol;

[0038] Step C2.3: M r Each k bits of data of ″′ is divided into one segment of data, and t segments of data are obtained;

[0039] Step C2.4: Count M″′ r The number of elements with the jth bit set to 1 in the t-segment data, where j = 1, 2, ..., k. If M″ r The number of elements with the jth bit set to 1 in the t-segment data is greater than or equal to Then let the j-th element of the watermark information M′ be 1, otherwise it is 0. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A watermark embedding flow chart of the implicit watermark method for the image generation model of the present invention;

[0041] Figure 2Flowchart of watermark extraction for the invisible watermark method of the image generation model of the present invention;

[0042] Figure 3 Structural diagram of the invisible watermark method of the image generation model of the present invention;

[0043] Figure 4 Illustration diagram of distorted attack picture samples for the invisible watermark method of the image generation model of the present invention;

[0044] Figure 5 Box plot of watermark extraction bit accuracy for the invisible watermark method of the image generation model of the present invention under different distortion attacks;

[0045] Figure 6 Average watermark bit accuracy graph of social media platforms for the invisible watermark method of the image generation model of the present invention under different distortion attacks;

[0046] Figure 7 Horizontal comparison graph of image quality and fidelity of multiple watermark algorithms for the invisible watermark method of the image generation model of the present invention under different distortion attacks. Detailed implementation manners

[0047] The present invention will be further described in detail below in conjunction with the embodiments in the accompanying drawings.

[0048] Embodiment 1: As shown in Figure 1 、 Figure 2 and Figure 3 , an invisible watermark method for an image generation model includes an invisible watermark embedding operation and an invisible watermark extraction operation. The specific process of the invisible watermark embedding operation is as follows: First, the original noise is unfolded one-dimensionally and binarized to obtain a binarized array of the original noise. The binarized array of the original noise is a one-dimensional array. The binarized array of the original noise is evenly segmented according to a fixed segment length, and the segment length is determined according to actual needs. The number of segments after segmentation is denoted as The watermark information in binary data form is expanded to obtain expanded watermark information with the number of data bits equal to , and the expanded watermark information is encrypted to obtain encrypted watermark data. The encrypted watermark data is a one-dimensional array, and its length is equal to Based on the encrypted watermark data, the binary array of the original noise is adjusted through parity matching to embed the encrypted watermark data, obtaining an adjusted binary array of the original noise. The adjusted binary array of the original noise is restored to obtain a restored noise that conforms to the Gaussian distribution. The statistical distribution characteristics of the restored noise are consistent with those of the original noise, and the watermark information is retained at the same time. Then, the restored noise is reshaped according to the size of the original noise to obtain a reshaped noise. First, an image generation model is used to generate reshaped noise data, and then an image decoder is used to generate an image from the reshaped noise, generating an image containing the encrypted watermark data; the specific process of extracting the hidden watermark is as follows: First, an image encoder is used to extract the noise containing the encrypted watermark data from the image containing the encrypted watermark data. Then, the noise containing the encrypted watermark data is unfolded in one dimension and binarized to obtain a binary noise array containing the encrypted watermark data. The length of the binary noise array containing the encrypted watermark data is equal to the length of the binary array of the original noise. The encrypted watermark data is extracted from the binary noise array containing the encrypted watermark data. Then, the extracted encrypted watermark data is decrypted to obtain the watermark data. Finally, error correction is performed on the watermark data to obtain the watermark information.

[0049] In this embodiment, by restoring the adjusted binary array of the original noise to a noise that conforms to the Gaussian distribution, the original noise maintains the statistical characteristics of the original noise, thus ensuring that the visual quality of the generated image is not affected by watermark embedding. During the watermark (encrypted watermark data) embedding process, only small and scattered local adjustments are made to the binary array, without changing the overall distribution characteristics of the original noise, thereby retaining the diversity of the image. Since the statistical characteristics of the restored noise are consistent with those of the original noise, watermark embedding does not affect the output fidelity and diversity of the image generation model.

[0050] Embodiment 2: This embodiment is basically the same as Embodiment 1, except that: in this embodiment, based on the encrypted watermark data, the specific process of adjusting the binary array of the original noise through parity matching to embed the encrypted watermark data is as follows: Let the th segment of data of the binary array of the original noise correspond to the th element of the encrypted watermark data, and respectively determine whether the sum of each segment of data in the binary array of the original noise and the one element in the encrypted watermark data corresponding to this segment of data are both odd or both even. If so, the segment of data in the binary array of the original noise remains unchanged. If not, the first element in the segment of data in the binary array of the original noise is flipped to achieve the adjustment of the binary array of the original noise, thereby obtaining an adjusted binary array of the original noise.

[0051] In this embodiment, when adjusting the binarized array of the original noise through parity matching, at most one bit is flipped each time, which has a negligible impact on the overall binarized array, making it difficult to perceive or detect the embedded watermark (encrypted watermark data). The image generation model is insensitive to small changes in the noise, and this fine-tuning can better retain the watermark information during the image generation process. Therefore, this watermark embedding method has a simple implementation process, low computational complexity and cost, high embedding efficiency, and is suitable for large-scale applications.

[0052] Embodiment 3: This embodiment is basically the same as Embodiment 2, except that: in this embodiment, the specific process of extracting the encrypted watermark data from the binarized noise array containing the encrypted watermark data is as follows: the binarized noise array containing the encrypted watermark data is evenly segmented, and the number of segments is also It is assumed that the encrypted watermark data to be extracted is a one-dimensional array, and the length is The th segment data of the binarized noise array containing the watermark data corresponds to the th element of the encrypted watermark data to be extracted. If the sum of the th segment data of the binarized array containing the watermark data is odd, then the th element of the encrypted watermark data to be extracted is assigned a value of 1. If the sum of the th segment data of the binarized noise array containing the watermark data is even, then the th element of the encrypted watermark data to be extracted is assigned a value of 0. Thus, the encrypted watermark data with a length equal to and being a one-dimensional array is extracted.

[0053] In this embodiment, the watermark extraction logic strictly corresponds to the watermark embedding logic, and the watermark can be losslessly restored without external interference. Moreover, during the watermark extraction process, only the sum of each segment of the binarized noise array containing the watermark data needs to be calculated and the parity is judged, without complex calculations, which is easy to implement and efficient.

[0054] Embodiment 4: This embodiment is basically the same as Embodiment 3, except that: in this embodiment, the specific process of expanding the original noise in one dimension and performing binarization processing to obtain the binarized array of the original noise is as follows:

[0055] S1. Denote the original noise as Z, denote the number of channels contained in the original noise Z as c, and denote the size of each channel of the original noise Z as h×w, where h represents the length and w represents the width. The two-dimensional arrays of each channel of the original noise Z are respectively flattened into one-dimensional arrays in row-major order, and then the one-dimensional arrays of all channels are sequentially concatenated in channel order to form a one-dimensional array with a length of c×h×w, denoted as Z f , denote Z fThe length is denoted as N, N = c × h × w, and Z is defined f [i] as the i-th element in Z f , where i = 1, 2, …, N;

[0056] S2. Binarize each element of the one-dimensional array Z f to obtain the corresponding binarized data. Among them, for the i-th element Z f in the one-dimensional array Z f [i], the corresponding binarized data is denoted as B z [i]. For the i-th element Z f in the one-dimensional array Z f [i], perform binarization conversion through formula (1) to obtain the corresponding binarized data B z [i]:

[0057]

[0058] where Φ(·) is the cumulative distribution function of the standard normal distribution;

[0059] S3. Use the binarized data corresponding to each element of the one-dimensional array Z f obtained in step S2 to form the binarized array B z of the original noise. Among them, B z is a one-dimensional array with a length of N. The i-th element of B z is B z [i].

[0060] In this embodiment, the original noise is unfolded one-dimensionally and binarized to obtain the binarized array of the original noise. The binarization operation retains the sign information of the noise, providing a basis for subsequent restoration. This process method is simple, applicable to any Gaussian distribution noise, and has low implementation cost.

[0061] Example 5: This example is basically the same as Example 4, except that in this example, the watermark information in binary data form is expanded to obtain the expanded watermark information with the number of data bits equal to , and the specific process of encrypting the expanded watermark information to obtain the encrypted watermark data is as follows:

[0062] A1. Set a positive integer l w (i.e., the number of segments, set according to actual needs) that can be divided evenly by N. Take every l z bits of elements in the binarized array B w of the original noise as a segment of data, so as to evenly divide the binarized array B z of the original noise into segments of data,

[0063] A2. Denote the watermark information in binary data form as M. M is a one-dimensional array. Denote the length of the watermark information M as k, where,

[0064] A3. Judge Whether it is an integer. If it is, repeat the watermark information M times to form the extended watermark information M r , M d = M, If not, repeat the watermark information M times and then fill in zeros at the end to form the extended watermark information M r , M p = M, p = 1, 2, …, is bit elements, and each element is 0, represents the ceiling symbol;

[0065] A4. Preset the key Key and encrypt the extended watermark information M r using the ChaCha20 stream cipher algorithm to obtain the encrypted watermark data M with a length of enc .

[0066] In this embodiment, the extension mechanism adapts to binary arrays of different lengths, with wide applicability. And by repeatedly embedding the watermark information, the redundancy is increased and the anti-noise ability is improved. The ChaCha20 stream cipher algorithm is efficient and secure, preventing the watermark from being accessed or tampered with without authorization.

[0067] Embodiment Six: This embodiment is basically the same as Embodiment Five, except that: In this embodiment, the specific process of restoring the adjusted binary array of the original noise to obtain the restored noise conforming to the Gaussian distribution is as follows:

[0068] B1. Denote the adjusted binary array of the original noise as B z ′. Denote the i-th element of B z ′ as B z ′[i], and denote the cumulative distribution function value of B z ′[i] as θ i , and calculate θ i using formula (2):

[0069]

[0070] where, u irepresents a random variable obtained by independently sampling from a uniform distribution defined on the closed interval [0, 0.5];

[0071] B2. Use the probability distribution function of the standard normal distribution to map θ i to obtain the corresponding Gaussian distribution value, denoted as Z f ′[i], as shown in formula (3):

[0072] Z f ′[i] = Φ -1 (θ i ) (3)

[0073] where Φ -1 (·) is the probability distribution function;

[0074] B3. Use the Gaussian distribution values obtained in step B2 to form the restored one-dimensional Gaussian noise array Z f ′, and the i-th element of Z f ′ is Z f ′[i]; reshape the restored one-dimensional Gaussian noise array Z f ′ according to the size of the original noise Z to obtain the restored noise Z′ that conforms to the Gaussian distribution.

[0075] In this embodiment, the restored noise distribution is consistent with the statistical characteristics of the original noise, ensuring that the image generation model receives similar inputs. By precise inverse CDF mapping (i.e., mapping of the probability distribution function of the standard normal distribution), the distribution deviation is reduced, guaranteeing the quality and diversity of the generated images.

[0076] Embodiment Seven: This embodiment is basically the same as Embodiment Six, except that: in this embodiment, the specific process of decrypting the extracted encrypted watermark data to obtain the watermark data and correcting the error of the watermark data to obtain the watermark information is

[0077] C1. Denote the extracted encrypted watermark data as M′ enc , and use the ChaCha20 stream cipher algorithm and the key Key to decrypt the extracted encrypted watermark data M e ′ nc to obtain the watermark data, denoted as M r ′, and the length of the watermark data M r ′ is

[0078] C2. Set the extended parameter ratio Judge whether Q is an integer. If it is, execute the complete segment error correction process to obtain the watermark information M′; if Q is a non-integer, execute the error correction process with truncation to obtain the watermark information M′;

[0079] The complete segment error correction process includes the following steps:

[0080] Step C2.1: Divide every k-bit element of the watermark data M r ′ into one segment of data, obtaining Q segments of data, with the length of each segment of data being k;

[0081] Step C2.2: Count the number of elements with the j-th bit being 1 in the Q segments of data of the watermark data M r ′ respectively, where j = 1, 2, …, k. If the number of elements with the j-th bit being 1 in the Q segments of data of the watermark data M r ′ is greater than or equal to then set the j-th bit element of the watermark information M′ to 1, otherwise to 0;

[0082] The error correction process including truncation includes the following steps:

[0083] Step C2.1: Set the maximum number of complete segments represents the floor symbol;

[0084] Step C2.2: Intercept the first t * k-bit elements of the watermark data M′ r and denote these t * l-bit elements as M″ r , where * is the multiplication operator;

[0085] Step C2.3: Divide every k-bit data of M″ r into one segment of data, obtaining t segments of data;

[0086] Step C2.4: Count the number of elements with the j-th bit being 1 in the t segments of data of M″ r , where j = 1, 2, …, k. If the number of elements with the j-th bit being 1 in the t segments of data of M″ r is greater than or equal to then set the j-th bit element of the watermark information M′ to 1, otherwise to 0.

[0087] In this embodiment, the majority voting mechanism can correct some errors, and can restore the watermark even if there is a distortion attack resulting in data corruption; the error correction process adapts to different extension situations to ensure the restoration effect; combining encryption and error correction ensures that the watermark is both secure and reliable.

[0088] To verify the performance of the invisible watermark method for the image generation model of the present invention, the following experiments are used for verification:

[0089] I. Experimental settings:

[0090] In this experiment, the code of the proposed method for embedding invisible watermark in image generation model was written in Python under the following hardware and software environments. Hardware: NVIDIA GeForce RTX 4090 graphics card; Software: Ubuntu 22.04 operating system.

[0091] The proposed method for embedding invisible watermark in image generation model was used to operate on the noise data, generating 1000 RGB images with a size of 512×512. Each image embedded 64-bit watermark information and was saved in PNG format. Subsequently, various distortion attacks and social media platform transmission tests were applied to these images, including WeChat Moments, WeChat chat, Xiaohongshu, Weibo, Bilibili, Twitter, and Instagram, to evaluate the robustness of the watermark. In addition, for comparison with existing methods, the Gaussian Shading (GS) method disclosed in reference [1] "Yang Z, Zeng K, Chen K, et al. Gaussian shading: Provable performance-lossless image watermarking for diffusion models [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2024: 12162-12171." and the TreeRing Watermark (TR) method disclosed in reference [2] "Wen Y, Kirchenbauer J, Geiping J, et al. Tree-rings watermarks: Invisible fingerprints for diffusion images [J]. Advances in Neural Information Processing Systems, 2023, 36: 58047-58063." were also used to operate on the noise data to generate 1000 RGB images with a size of 512×512, and their image fidelity and diversity were evaluated. The implementation of the two existing methods was also completed under the above hardware and software conditions.

[0092] II. Experimental Metrics:

[0093] The accuracy of watermark extraction is measured by bit accuracy, which is defined as the ratio of the number of correctly extracted bits to the total number of bits. The value range of this metric is from 0% to 100%. When the bit accuracy is close to 100%, it indicates that the watermark information is effectively extracted; when the bit accuracy is close to 50%, it indicates that the watermark information fails to be extracted and is close to the random guessing level; in extremely rare cases, if the bit accuracy is close to 0%, the watermark information can be restored by inverting each bit, but this situation rarely occurs in practice. This experiment uses this metric to evaluate the robustness of the watermark after distortion attacks and social media transmission.

[0094] To evaluate the visual quality and diversity of the generated images, the following metrics are used: FID (Fréchet Inception Distance): Measures the similarity between the distribution of the generated images and the reference images. The lower the FID value, the closer the distribution of the generated images is to the reference distribution, and the smaller the impact of watermark embedding on the image quality. CLIP-FID: A FID metric that combines the CLIP model to evaluate the consistency between the generated images and the text prompts. The lower the CLIP-FID value, the higher the matching degree between the image content and the prompts, and the smaller the impact of watermark embedding on the image fidelity. KID (Kernel Inception Distance): Evaluates the difference between the distribution of the generated images and the reference images. The lower the KID value, the closer the distribution of the generated images is to the reference images, and the smaller the negative impact of the watermark method on the image diversity.

[0095] III. Experimental operation process:

[0096] 1. Simulation of distortion attacks: Apply the following distortion attacks to 1000 watermarked images generated by the proposed invisible watermark method for image generation models and two existing methods respectively, with each attack using different intensities: Gaussian noise, Gaussian blur, JPEG compression, image size reduction, random position cropping of the image, random loss of the image, contrast enhancement, brightness enhancement, image regeneration attack using variational autoencoder (VAE). After that, perform watermark extraction operations on the images and calculate the bit accuracy. Examples of specific distortion attacks are Figure 4 as shown.

[0097] 2. Image Fidelity and Diversity Tests: To compare the image quality of the proposed invisible watermarking method for image generation models with two existing methods, the following test groups were set up: Reference Group (G1-No-W): 1000 512×512 RGB images without watermarks. Control Group (G2-No-W): 1000 512×512 RGB images without watermarks. Method Group of the Present Invention (G2-W-Ours): 1000 512×512 RGB images with 64-bit watermarks. GS Method Group (G2-W-GS): 1000 512×512 RGB images with 64-bit watermarks, using the Gaussian Shading method (Reference [1]). TR Method Group (G2-W-TR): 1000 512×512 RGB images with 64-bit watermarks, using the TreeRing Watermark method (Reference [2]). Using FID, CLIP-FID, and KID metrics, the image distribution differences were calculated between the reference group (G1-No-W) and the control group (G2-No-W), and between the control group (G2-No-W) and each watermark method group (G2-W-Ours, G2-W-GS, G2-W-TR). To visually compare the fidelity, four types of images (G2-No-W, G2-W-Ours, G2-W-GS, G2-W-TR) were generated using the same prompt (such as "natural scenery") for horizontal comparison.

[0098] 3. Social Media Platform Tests: The watermarked images were uploaded to social media platforms respectively, all the uploaded pictures were downloaded and watermark extraction was performed. After watermark extraction, the bit accuracy was calculated.

[0099] IV. Analysis of Experimental Results:

[0100] 1. Simulation of Distortion Attacks: Under the condition of no attack, based on the statistical results of 1000 test images, the watermark extraction bit accuracy of the proposed invisible watermarking method for image generation models was 100%. In most distortion attacks, even when the attack intensity was high, the bit accuracy remained above 90%, showing good robustness. Only when the attack intensity was extremely high (such as severe image distortion), the bit accuracy dropped to around 50%, and at this time the image could no longer be used normally. The results are as Figure 5 shown. Analysis Figure 5 shows that the proposed invisible watermarking method for image generation models maintains stable watermark extraction accuracy under mild to moderate attacks, proving its excellent anti-attack ability.

[0101] 2. Horizontal Comparison with GS and TR Methods:

[0102] Fidelity Comparison: Horizontal comparison was performed on the images generated using the same prompt. The results are as Figure 6 shown. AnalysisFigure 6 It can be seen that the images (G2-W-Ours) generated by the invisible watermarking method for the image generation model of the present invention are highly consistent in visual effect with the control group (G2-No-W), while the images generated by the existing two methods are significantly different from the control group, proving that the invisible watermarking method for the image generation model of the present invention has significant advantages in fidelity.

[0103] Diversity comparison: Evaluated by FID, CLIP-FID, and KID metrics, the results are shown in Table 1, where G1-No-W is the watermark-free reference group and G2-No-W is the watermark-free control group.

[0104] Table 1: Comparison of image diversity metrics

[0105] Dataset FID↓ CLIP - FID↓ KID↓ G1 - No - W v.s.G2 - No - W 24.46 1.012 0.00058 G1 - No - W v.s.W - TR 41.37 3.269 0.01978 G1 - No - W v.s.W - GS 38.97 3.001 0.01389 G1 - No - W v.s.W - Ours 24.98 0.958 0.00037

[0106] Analyzing Table 1, it can be seen that the FID, CLIP-FID, and KID values of the method of the present invention (G2-W-Ours) are close to those of the control group (G2-No-W), and are much lower than those of the GS and TR methods, indicating that the impact of watermark embedding on image diversity is minimal and is superior to existing methods.

[0107] 3. Social media platform test: The test results on the social media platform are as Figure 7 shown. Figure 7 shows the average value of the watermark bit accuracy on different platforms, in percentage. Figure 7 The test results show that after uploading and downloading pictures on the social media platform, the bit accuracy of the watermark information remains above 99%, showing extremely high robustness and effectiveness. Especially on common platforms such as WeChat and Weibo platforms, the watermark bit accuracy is close to 100%, indicating that there is almost no loss of watermark information on these platforms. Generally speaking, all test platforms have demonstrated the high accuracy and stability of the invisible watermarking method for the image generation model of the present invention in extracting watermark information in the actual social platform, verifying the effectiveness of the invisible watermarking method for the image generation model of the present invention in practical applications.

[0108] In summary, the invisible watermarking method for the image generation model of the present invention can effectively embed watermarks while ensuring the quality of the generated images, and can also ensure the diversity of the generated images, without having a negative impact on the output fidelity and data true representation of the image generation model. It has broad application prospects in the protection of the originality and copyright tracking of the images generated by the image generation model.

Claims

1. A method for embedding invisible watermarks in an image generation model, including an invisible watermark embedding operation and an invisible watermark extraction operation, characterized in that: The specific process of the hidden watermark embedding operation is as follows: First, the original noise is unfolded one-dimensionally and binarized to obtain a binarized array of the original noise. The binarized array of the original noise is a one-dimensional array. The binarized array of the original noise is evenly segmented according to a fixed segment length, and the number of segments after segmentation is denoted as The watermark information in binary data form is expanded to obtain expanded watermark information with the number of data bits equal to The expanded watermark information is encrypted to obtain encrypted watermark data. The encrypted watermark data is a one-dimensional array, and its length is equal to Based on the encrypted watermark data, the binarized array of the original noise is adjusted through parity matching to embed the encrypted watermark data, obtaining an adjusted binarized array of the original noise. The adjusted binarized array of the original noise is restored to obtain a restored noise that conforms to the Gaussian distribution. The statistical distribution characteristics of the restored noise are the same as those of the original noise, and the watermark information is retained at the same time. Then, the restored noise is reshaped according to the size of the original noise to obtain a reshaped noise. First, an image generation model is used to generate reshaped noise data, and then an image decoder is used to generate an image from the reshaped noise data, generating an image containing the encrypted watermark data; The specific process of the hidden watermark extraction is as follows: First, an image encoder is used to extract the noise containing the encrypted watermark data from the image containing the encrypted watermark data, and then the noise containing the encrypted watermark data is unfolded one-dimensionally and binarized to obtain a binarized noise array containing the encrypted watermark data. The length of the binarized noise array containing the encrypted watermark data is equal to the length of the binarized array of the original noise. The encrypted watermark data is extracted from the binarized noise array containing the encrypted watermark data, and then the extracted encrypted watermark data is decrypted to obtain the watermark data. Finally, the watermark data is corrected to obtain the watermark information.

2. The implicit watermarking method for an image generation model according to claim 1, characterized in that Based on the encrypted watermark data, the specific process of adjusting the binary array of the original noise through parity matching to embed the encrypted watermark data to obtain the adjusted binary array of the original noise is as follows: Let the segment data of the binary array of the original noise correspond to the bit element of the encrypted watermark data. Respectively judge whether the sum of each segment data of the binary array of the original noise and the one-bit element corresponding to this segment data in the encrypted watermark data are both odd or both even. If so, the segment data of the binary array of the original noise remains unchanged. If not, flip the first-bit element in the segment data of the binary array of the original noise to achieve the adjustment of the binary array of the original noise, and thus obtain the adjusted binary array of the original noise.

3. The method for embedding a watermark into an image generation model according to claim 2, wherein The specific process of extracting encrypted watermark data from a binarized noise array containing encrypted watermark data is as follows: The binarized noise array containing encrypted watermark data is evenly segmented, and the number of segments is also It is assumed that the encrypted watermark data to be extracted is a one-dimensional array, and the length is The th segment data of the binarized noise array containing watermark data corresponds to the th element of the encrypted watermark data to be extracted. If the sum of the th segment data of the binarized array containing watermark data is odd, then the th element of the encrypted watermark data to be extracted is assigned a value of 1. If the sum of the th segment data of the binarized noise array containing watermark data is even, then the th element of the encrypted watermark data to be extracted is assigned a value of 0. Thus, the encrypted watermark data with a length equal to and being a one-dimensional array is extracted.

4. A method for embedding watermark in an image generation model according to claim 3, wherein The specific process of expanding the original noise in one dimension and binarizing it to obtain the binarized array of the original noise is as follows: S1. Denote the original noise as Z, denote the number of channels included in the original noise Z as c, and denote the size of each channel of the original noise Z as h×w, where h represents the length and w represents the width. Flatten the two-dimensional array of each channel of the original noise Z into a one-dimensional array in row-major order, and then splice the one-dimensional arrays of all channels in channel order to form a one-dimensional array with a length of c×h×w, denoted as Z f , denote the length of Z f as N, N = c×h×w, and define Z f [i] as the i-th element in Z f , where i = 1, 2, …, N; S2. Binarize each element of the one-dimensional array Z f to obtain the corresponding binarized data, where the i-th element Z f of the one-dimensional array Z f [i] has the corresponding binarized data denoted as B z [i], and the i-th element Z f of the one-dimensional array Z f [i] is binarized through formula (1) to obtain the corresponding binarized data B z [i]: Among them, Φ(·) is the cumulative distribution function of the standard normal distribution; S3. Use the one-dimensional array Z obtained in step S2 f The binarized data corresponding to each element forms the binarized array B of the original noise z , where B z is a one-dimensional array with a length of N, and the i-th element of B z is B z [i].

5. A method for embedding watermark in an image generation model according to claim 4, wherein Expand the watermark information in binary data form to obtain the expanded watermark information with the number of data bits equal to The specific process of encrypting the expanded watermark information to obtain the encrypted watermark data is as follows: A1. Set a positive integer \(l\) that can be divided evenly by \(N\). w , and for the binary array \(B\) of the original noise z , take every \(l\) w elements as a segment of data, thereby evenly dividing the binary array \(B\) of the original noise z into segments of data. A2. Denote the watermark information in the form of binary data as M. M is a one-dimensional array. Denote the length of the watermark information M as k, where, A3. Judgment Check if it is an integer. If so, repeat the watermark information M times to form the extended watermark information M r , M d = M, If not, repeat the watermark information M times and then pad zeros at the end to form the extended watermark information M r , M p = M, is a bit element, and each bit element is 0, represents the ceiling symbol; A4. Preset a key Key, and use the ChaCha20 stream cipher algorithm to encrypt the extended watermark information M r to obtain an encrypted watermark data M with a length of enc .

6. A method for embedding watermark in an image generation model according to claim 5, characterized in that The specific process of restoring the adjusted binarized array of the original noise to obtain the restored noise conforming to the Gaussian distribution is as follows: B1. Denote the adjusted binary array of the original noise as B'. z , and denote the i-th element of B' z as B' z [i]. Denote the cumulative distribution function value of B' z [i] as θ i . Calculate θ i using formula (2): where, u i represents a random variable obtained by independently sampling from a uniform distribution defined on the closed interval [0, 0.5]; B2. Use the probability distribution function of the standard normal distribution to map θ i to obtain the corresponding Gaussian distribution value, denoted as Z′ f [i], as shown in formula (3): Z′ f [i] = Φ -1 (θ i ) (3) where, Φ -1 (·) is the probability distribution function; B3. Use the Gaussian distribution values obtained in step B2 to form the restored one-dimensional Gaussian noise array Z'. f , Z' f The i-th element of f is Z' f [i]; Reshape the restored one-dimensional Gaussian noise array Z' f according to the size of the original noise Z to obtain the restored noise Z' that conforms to the Gaussian distribution.

7. A method for embedding watermark in an image generation model according to claim 6, wherein The specific process of decrypting the extracted encrypted watermark data to obtain the watermark data and correcting the errors of the watermark data to obtain the watermark information is C1. Denote the extracted encrypted watermark data as M'. enc , and use the ChaCha20 stream cipher algorithm and the key Key to decrypt the extracted encrypted watermark data M'. enc to obtain the watermark data, which is denoted as M'. r , and the length of the watermark data M' r is C2. Set the extended parameter ratio Determine whether Q is an integer. If so, execute the complete segment error correction process to obtain the watermark information M'. If Q is not an integer, execute the error correction process with truncation to obtain the watermark information M'; The complete segment error correction process includes the following steps: Step C2.1: Divide every k-bit element of the watermark data M′ r into a segment of data, obtaining Q segments of data, each segment having a length of k; Step C2.2: Statistically analyze the watermark data M r ′ respectively to obtain the number of elements with the value of 1 in the j-th bit of the Q-segment data, where j = 1, 2, …, k. If the number of elements with the value of 1 in the j-th bit of the Q-segment data of the watermark data M′ r is greater than or equal to then set the j-th bit element of the watermark information M′ to 1; otherwise, set it to 0. The error correction process with truncation includes the following steps: Step C2.1, set the maximum number of complete segments represents the floor symbol; Step C2.2: Intercept the watermark data M′ r Take the first t*k elements, and denote these t*k elements as M″ r , where * represents the multiplication operator; Step C2.3: Divide every k bits of M″ r into one data segment, obtaining t data segments; Step C2.4, count M″ r the number of elements with the value of 1 in the j-th bit of the t segments of data, where j = 1, 2, …, k. If the number of elements with the value of 1 in the j-th bit of the t segments of data of M″ r is greater than or equal to then set the j-th bit element of the watermark information M′ to 1, otherwise to 0.

Images